Concordia University ACBSP Self-Study Standards 3 and 4 ......Portland Business Community Exposure to best practices in community well-being, sustainability, and social entrepreneurship.

1 Concordia University SOM – ACBSP Self-study July, 2013

Concordia University

ACBSP Self-Study Standards 3 and 4

July, 2013


Standard 3 – Student and Stakeholder Focus

Criterion 3.1 – Student Segments Targeted and served student segments within the School of Management include prospective students, undergraduate students and graduate students.

Criterion 3.2 – Stakeholders Other than students, the major stakeholders include the School of Management Alumni, Internship sponsors/employers, the business community of Portland, and full-time and part-time faculty and staff. The primary method of gathering relevant information from students and stakeholders is through the use of surveys. Prospective students are surveyed by the Admissions department and the Office of Institutional Assessment each year. Newly enrolled undergraduate and graduate students are surveyed during orientation. Students in both the undergraduate and graduate programs are surveyed following each course taken with an individual course assessment (Exhibit 5d). Furthermore, graduating students in both the graduate and undergraduate programs are given an end of program evaluation which assesses their satisfaction with their entire program’s outcomes. Other stakeholders are also issued periodic surveys to gauge their expectation and satisfaction with our offered programs and services including the Alumni, School of Management Faculty and Staff, Internship sponsors/employers, and the Portland business community.

Criterion 3.3 – Listening and Learning Methods As a Christian and student-centric organization, the SOM pays extreme attention to student satisfaction. Review of the instructional methods used in SOM courses, and their relative effectiveness, is part of each Fall Faculty Retreat agenda. Faculty spend time each year sharing best practices and new approaches they have used in their classrooms, the response they received from students, and developing plans for updating their instructional approaches collaboratively. Because the Faculty Retreat is also the time when data from student and stakeholder surveys, as well as student outcomes, is shared, the data and the qualitative experiences of faculty both contribute to the design of new approaches. See Exhibit 26 – SOM Fall12 Retreat Agenda for an example of a Fall Retreat meeting agenda.

Over the past two years, the following adjustments to instructional approach have been made as an outcome of this annual reflection process:

• BA program approach was reviewed and revised to emphasize the development of learning communities and applied experiences in local Portland businesses. This approach was introduced in Fall 12 by Dr. Jones and Professor Rikert.

• Increased emphasis on case study method with MBA case study training provided by Professor Rikert (a graduate of Harvard and experienced Harvard case study facilitator).

• An additional option for the MBA capstone (Baldrige organizational assessment) was proposed by Dr. Gracianette in response to student need for an option to the business plan capstone. To be included beginning in Spring 14.

• MBA meeting frequency for future cohorts increased from two weekends per course to three weekends per course in order to increase contact time with students and ensure that course expectations and assignments are well supported early in the course. New meeting frequency scheduled for Fall 13.


• We also took a close look at MBA student’s profiles and observed a growing disparity between the level of maturity, professional experience and communication skills of American students, and the younger age, limited professional experience and inconsistent levels of English proficiency of international students. As a result, two additional MBA delivery options (weeknight MBA and online MBA) were proposed to the faculty in order to expand options for students and address the needs of 1) students who need more frequent meetings than a weekend format will allow; and 2) students who need the freedom of asynchronous learning supported by online technologies. New options to be offered beginning in Spring 14.

• Similar data on student outcomes and satisfaction for the BA program was reviewed as part of the self-study. For the most part, the data showed a stable level of student achievement and satisfaction, which led to the decision to dig deeper into the instructional approaches and curriculum design in the BA programs.

• The outcome of the BA study is presented in Exhibits 24a and 24b. A review of the program redesign and recommendations is underway at Concordia at the time of this writing (Summer 13). Pursuant to the review, the program redesign will commence in AY 2013-14, with the new program re-launched in AY 2014-15.

Criterion 3.4 – Using Stakeholder Information In addition to reviewing survey data when it is received (at the end of each course and at graduation for example), the School of Management conducts regular reviews of all student and stakeholder survey data at its annual Fall Retreat. This data is used (along with student outcomes data) to make adjustments to the curriculum and teaching methods, and to provide general guidance for the School of Management in its annual continuous improvement and planning process.

The School of Management also conducts regular reviews of programs with faculty in order to broaden the range of input to include faculty experience with students and stakeholders as well as research into industry trends. Exhibit 15 – MBA Townhall Meeting Minutes – is an example of notes from a meeting held specifically to gather input on the MBA program delivery mechanisms and concentrations from faculty in January, 2013.

Figure 3.1 – Table for Student and Stakeholder Groups

Student/Stakeholder Group Student/Stakeholder Requirement Educational Program Addressing Requirement

Regional Population Understanding of Concordia’s mission, vision, and programs

Marketing and Brand campaigns. Institutional Brand Survey to assess results.

Prospective Students Information about the benefits and opportunities afforded by completing an undergraduate or graduate degree in the School of Management.

Marketing materials and recruiting outreach to discuss program. The annual MBA wine and cheese event for prospective MBA students.

Current Students – Undergraduate Overview of the BA Program, requirements, policies, courses, faculty, and staff

New Student Orientation at the beginning of each Semester (with accompanying survey) Monthly newsletter

Current Students - Graduate Overview of the MBA Program, requirement, policies, courses, faculty, and staff

New Student Orientation at the beginning of each cohort launch


Alumni Success after graduation Programs that contain course work and internship / capstone experiences for relevant business application. Alumni survey used to assess success after graduation.

Internship sponsors/employers Well prepared students to learn through experiential internships and potentially fill positions after graduation.

Internship courses, individual coaching and counseling, focus on matching intern with employer for optimum “fit.” Assessed through internship surveys.

Portland Business Community Exposure to best practices in community well-being, sustainability, and social entrepreneurship.

TEDx Event

Criterion 3.5 – Attracting and Retaining Students Admission and Retention Policies are set by the Office of the Provost and Admissions Department, and are documented in the Concordia University Catalog on pages 31-41 for undergraduate and pages 200-204 for the MBA. In addition, the following policies and procedures are maintained by the School of Management for the MBA program:

http://www.cu-portland.edu/som/mba/admissions.cfm (MBA admission & app checklist) http://www.cu-portland.edu/som/mba/documents/MBA_International.pdf (MBA International admission & app checklist) http://www.cu-portland.edu/catalog/graduate_education/som/admission.cfm (MBA Admission criteria/process)

All admission policies are recommended by the faculty and approved by the Board of Regents and administered by the Office of the Provost and the Admissions Office. There is basically no transfer credit allowed to substitute for the core MBA curriculum. All students must successfully complete it. In the past, credits for the six hours of elective credits were permitted from an approved graduate program usually in a specific area (e.g., finance). Because of our current emphasis on Asian Business, Internships/Thesis Projects, and Healthcare Administration (HCA), allowable transfer credit is very minimal.

The SOM’s strategic goals are focused on quality growth that aligns with the broader university strategy: Vision 2020. While a number of processes for driving and retaining students are managed by SOM, the majority are jointly driven and resourced by the SOM and centralized services such as the Admission’s office collaboratively. This structure insures tight, university wide alignment to Vision 2020 and preserves budget to be fairly allocated to initiatives and support activities. Therefore, SOM works in partnership with Concordia’s centralized services for student recruitment (admissions), counseling / student services, career services, marketing, Concordia Foundation (for community, donors, business relationships) and alumni services to attract, retain and build / enhance stakeholder relationships.

Figure 3.1a highlights those institutional departments and functions that efficiently and effectively work with the School of Management to admit and retain students in addition to stakeholder support.

http://www.cu-portland.edu/som/mba/admissions.cfm

http://www.cu-portland.edu/som/mba/documents/MBA_International.pdf

http://www.cu-portland.edu/catalog/graduate_education/som/admission.cfm


Several of these centralized departments manage budgets completely dedicated to the support of SOM student recruitment activities. The budget amount for the year is determined by the Provost after reviewing each school / college’s annual strategic plan and student enrollment goals.

Specific activities involving SOM allocated funds require the review of the SOM Dean. While the centralization of these functions into one service group for the entire university is an efficient organization structure, the effectiveness of the allocated budget is insured through the review and sign off of the Dean on budget spending.

Many of the strategic plan success metrics (student enrollment, program enrollment, student retention, active alumni percentage, post graduate employment rate, student four year graduation rate) are shared both by SOM for their particular sub population of the university, and by centralized departments (admissions, student services, alumni services). The sharing of success metrics also insures successful integration

In addition to the activities and processes that support the broader population of Concordia students, School of Management has several key internal processes to attract, retain students and engage key stakeholders:

• Virtual “Open Office” policy. In addition to physical office hours, the Dean and SOM administrative staff have a “24 hour or less response” to student e-mails, voice mails and texts. Responsiveness, even if to just acknowledge to the student that their issue has been received, and an estimate of when a solution will be found – is a key factor in students feeling valued by a school or college.

• Hosting and sponsorships of Oregon FBLA and DECA organizations. FBLA and DECA represent key high school organizations that groom students to pursue college degrees in business. Each year, Concordia SOM hosts each group at Concordia which include presentations by SOM about our school and programs, and opportunity to make follow up visits to high schools to promote Concordia School of Management.

• Each July and January, the Dean and staff review key metrics of enrollment by course / program, retention rates by SOM undergraduate and graduate tracks, and student course / faculty evaluations. The goal for all these metrics is for continuous improvement, and only by reviewing the latest trend in the data can appropriate actions and initiatives be assigned and tracked to drive additional improvement.

• Allocation and management of SOM discretionary funds. The SOM Dean has at their disposal a discretionary budget for hospitality (luncheon or reception events for students and business leaders), travel (to meet with high school, community college students and advisory councilors), memberships (for local business professions organizations, student club chapters), honorariums (for business advisory boards, guest speakers), student workers, and misc. funds for student travel to local businesses, supplies for events, etc. These funds are reviewed each month with program chairs and department heads to insure SOM is building brand and loyalty with both students, community (prospective students and employers) and key stakeholders. Faculty meetings are an additional venue to review these events and requests for future events, activities, memberships etc.


Figure 3.1a Institution Centralized Departments and Role, Operations in Support of School of Management Goals Institution

Department Student /

Stakeholder Primary Focus

Information, data reporting to SOM?

SOM process signoffs?

Activities in support of SOM strategic plan goals

SOM – Department Scheduled Meetings

Admissions Student recruiting - Weekly recruiting funnel - Weekly new student acceptance / enrollment

- Admission criteria (MBA) - MBA student interviews

- Quarterly open house events for undergraduates including Dean panel - Three times / year MBA recruiting events - Quarterly SOM radio spots - Paid Search for SOM undergraduate program recruitment - Community college degree transfer agreements

- Monthly ops meeting - Semiannual planning meeting

Student services

Student retention - Grievance filing - Drop out / transfer out by reason (semi annual report) - academic probation

- Incompletes / extensions - Directed studies - Course offering schedule changes

- Orientation sessions for SOM to meet new students - Business Club sponsorship, awareness fair each semester

- Monthly review of student issues - Attendance at SOM faculty meetings

Career Services

Student, graduate employment

- Posting of internships, part time and full time jobs

Partnership with SOM Faculty

- Business leader panel on how to get entry jobs at their companies

Internship classes and individual meetings with students

Marketing Student prospects, stakeholders

- Monthly SOM web activity reports - Monthly SOM related activity reports

- Any SOM collateral (brochures, ads) - SOM web pages

- New SOM offering launch meetings for collateral development, outreach plans - Campaign development meetings and plans for key SOM strategic plan goals

- Monthly review of outreach, web activity - Semi-annual planning

Foundation Stakeholders (donors, employers)

- invitation (Dean) to events including business leaders

Alumni Services

Graduates SOM graduate contact information

SOM facilitates updates to contact info each semester

- E-mail outreach for announcing new SOM graduate programs and seminars - Website focus on SOM alumni stories

- Semi-annual planning meeting for SOM outreach on seminars, programs

President’s Office

Stakeholders (donors, employers)

- Invitations to attend events, meetings with stakeholders - Invitations to trustee, board meetings

- Invitations to present SOM strategy at Trustee meetings, - Arrange SOM interviews with local press - “Seats” at various community events for business leaders, business awards

As needed


Criterion 3.6 – Seeking Information The School of Management proactively gathers information such as recommendations, complaints, requests from students and stakeholders through:

• Formal course, instructor and program evaluations. Each semester, each faculty member, program director, department head and the Dean receive appropriate levels of student course and instructor feedback. The Dean meets with the program directors to review feedback for each program, and creates action plans based on that review. Actions can range from adjustments to syllabus, to counseling of an instructor.

• Regularly scheduled meetings with university stakeholder departments and program advisory boards. School of Management leverages regularly scheduled meetings with admissions, SOM faculty, student services, academic council, and externally sourced advisory boards to secure feedback on new initiatives, program and college core metrics, and annual strategic plan. Structured and unstructured feedback serves to shape SOM strategic plans, initiatives, program changes, course updates, activities, admissions criteria and hiring practices.

• Grievance process for student complaints. Concordia University has adopted a formal “grievance” policy and process for management of complaints from students. If a student has an issue with an instructor, a grade, policy or course (curriculum, learning outcome coverage, time required etc.) the process clearly includes notification and involvement of the Dean of the college where the grievance occurs. Grievances are documented and reviewed for situation resolution, root cause analysis, and systemic approach for preventing future incidences.

• Topic specific SOM surveys. Throughout an academic year, program directors, department heads and SOM faculty deploy surveys to students and stakeholders for feedback ranging in topics from interest in an event or speaker to proposed schedule changes. Surveys are especially useful to normalize feedback from a small group or single stakeholder.

• Communication Tools. School of Management utilizes program newsletters, LMS bulletin boards and e-mail lists to communicate upcoming events, speakers, seminars, special interest meetings, internships and business information nights, business area competitions or sponsorships. Responses and attendance guide SOM in determining common interests and activities across a range of students and stakeholders.

It is the culture at SOM to be “customer focused” (students, parents, employers, partners and internal / external stakeholders) when facing a decision, thinking about a new idea or change, or handling an issue. One of Concordia’s three core values is servant leadership. That value is manifested by continually checking in on the impact and perceptions of students and stakeholders of SOM’s faculty, programs and strategic actions. As a faith based institution, “walking the talk” of servant leadership is embraced and modeled at the core of Concordia administration, faculty and staff. The SOM’s strategy to deal with complaints from students and stakeholders is to handle all issues through promptness, careful listening, caring and sincerity. We also look for root causes associated with reported issues, and compare them to stated university and SOM policy. Our Christian heritage allows us to carefully listen to students’ or stakeholders’ reported issues with great patience, caring and sincerity in managing such issues.

Procedures for student grievances (e.g., concerns over grades, grading policies, grading practice, course design) are delineated in the Student Handbook, available at http://intranet.cu-portland.edu/students/documents/StudentHandbook.pdf. These procedures are designed to provide a

http://intranet.cu-portland.edu/students/documents/StudentHandbook.pdf

http://intranet.cu-portland.edu/students/documents/StudentHandbook.pdf


mechanism by which students can seek to express concerns, disagreements, or complaints about a faculty member, and seek redress of perceived injustice, harassment, discrimination, or inequity. Students are encouraged to start by discussing concerns face-to-face with the faculty member in an attempt to resolve issues. Students are also encouraged to discuss concerns with their academic advisor or a professional staff person in Student Services. This provides an opportunity to review the appropriate university policies and practices related to the student’s concern.

Criterion 3.7 – Student and Stakeholder Satisfaction The School of Management system for monitoring student and stakeholder satisfaction is detailed in Figure 3.2

Figure 3.2 – Table for Student and Stakeholder Satisfaction Student/Stakeholder Group Satisfaction Measure Dissatisfaction Measure Regional Population 60% of regional population indicates

Concordia lives up to its mission Less than 50% of regional population indicates Concordia lives up to its mission

Prospective Students 60% of prospective students indicate that Concordia’s mission attributes are important to their choice of colleges.

Less than 50% of prospective students indicate that Concordia’s mission attributes are important to their choice of colleges.

Current Students 80% of current students indicate Concordia lives up to its mission

Less than 60% of current students indicate Concordia lives up to its mission

Alumni 80% of alumni indicate degree has been beneficial to their career

Less than 60% of alumni indicate degree has been beneficial to their career

Internship sponsors/employers

80% of interns are rated at the “B” level or above by internship sponsors

Less than 60% of interns are rated at the “A+” or “A” level by internship sponsors

Portland Business Community 90% positive response to the overall TEDx event

Less than 70% positive response to the overall TEDx event


Criterion 3.8 – Student and Stakeholder Focused Results The business unit should present graphs or tables of assessment results pertinent to this standard.

Performance Measure

Instrument Current Results

Analysis of Results

Action Taken/Planned

Graphs or Tables of Resulting Trends

60% of Regional Population indicate Concordia lives up to its mission

Institutional Brand Survey (bi-annual)

Overall awareness and perception of Concordia is up since 2010.

Marketing and brand awareness efforts are working.

Continue to promote Concordia’s mission and vision with external stakeholders and repeat survey annually.

60% of Prospective Students agree that Concordia’s mission attributes are “very much” or “somewhat” important to their choice of colleges.


Overall agreement to key mission attributes with slight declines in value assigned to “servant leaders” and “spiritual development” since 2010

Marketing and brand awareness efforts are working.

Continue to promote Concordia’s mission and vision with external stakeholders and repeat survey annually.


Performance Measure


Analysis of Results

Action Taken/Planned


80% of Current Students indicate Concordia lives up to its mission


Overall perception of Concordia by current students is slightly up since 2010 with exception of specific questions on graduate attributes.

Marketing and brand awareness efforts are working, but should receive increased focus internally.

• Continue to promote Concordia’s mission and vision with internal stakeholders and repeat survey annually.

• Add question to separate survey data by degree program in future years. Target SOM students as indicated by survey results.

60% response rate to End of Course Assessments by Current Students

End of Course Assessment

88% of evaluations completed in Fall 12 46% in Spring 13

Conducted paper-based assessments in the classroom in Fall 12 to increase response rate. Returned to online assessments in Spring 13, with increase in response from 34% to 46% since last Spring.

• Continue with online assessments, maintaining goal of a 60% response rate.

• Emphasize the importance of the assessments to faculty and students.

• Send out personal reminders from Dean and Program Directors at the end of each course.

• Send an annual summary to students of changes made to programs and courses based on their feedback.

25% 35% 34%

88%

46%

0%10%20%30%40%50%60%70%80%90%

100%

Response Rate by

Percentage

Term Surveyed

End of Course Evaluation - Response Rates Spring 2011 to Spring 2013

Response Rate


Performance Measure


Analysis of Results

Action Taken


Gather data on the needs, interests, and activities of Current Students

Undergrad Orientation Survey (annual)

In 2011 and 2012 over 50% of students participate in athletics.

We continue to have a high number of student athletes and large proportion of students who are residents.

• Add a Sports Management degree to address interests of student athletes in Portland (market rich with sporting goods companies).

Seek input on potential electives in new MBA from Current Students

2013 MBA Student Survey (Exhibit 22)

Students prioritized choice of concentrations. This data was reviewed, along with other data from industry stakeholders, to design the elective portion of the revised MBA.

• Selected top four topics of interest to MBA students when designing the concentrations for the revised MBA.


Performance Measure


Analysis of Results

Action Taken / Planned


80% of Graduates indicated overall satisfaction with their program

Undergrad End of Program Survey (annual)

More than 80% of students have responded positively since Spring 2011.

Meeting goal

• Emphasize the value of the business degree to students.

• Support students with internships.

• Continue to monitor trends.

80% of Graduates indicated overall satisfaction with their program

MBA End of Program Survey (annual)

100% of students responded positively in Spring 2013 (Cohort 22)

Meeting goal

Feedback from Cohort 20 was used to modify the program by new Dean and MBA Program Director who started with SOM in Fall 12. Survey results in three most recent cohorts (6-Bend, 21-PDX, and 22-PDX) show evidence of increasingly positive response by students.

82 88 87

05

101520253035404550556065707580859095

100

Spring 2011 Spring 2012 Spring 2013

% o

f Stu

dent

s who

said

Yes

Semester

Would you recommend Concordia BA program to a friend?

Yes

100 100 89

60

88 86

100

05

101520253035404550556065707580859095

100

Cohort18 - PDX

Cohort19 - PDX

Cohort5 - Bend

Cohort20 - PDX

Cohort6 - Bend

Cohort21 - PDX

Cohort22 - PDX

% o

f stu

dent

s who

said

Yes

Would you recommend the CU MBA program to your colleagues?

Percent


Performance Measure


Analysis of Results



80% of Alumni indicate degree has been beneficial to their career

BA Alumni Survey (bi-annual)

37 out of 53 students (70%) surveyed in 2012 indicated “satisfied” or “extremely satisfied”

Failing to meet this measure.

• This data was used to revamp BA program beginning in Fall 12.

• Continue to monitor this data in response to changes made.

• Add a question to allow separating this data by graduating class to track change in response based on program updates.

Spring 2012 BA Alumni Survey

80% of Alumni indicate degree has been beneficial to their career

MBA Alumni Survey 2011 (bi-annual)

83% of alumni indicated that the internship or thesis helped them achieve career goals.

Meeting this measure.

• Add more specific career accomplishment questions to future alumni surveys.

• Add ability to separate this data by graduating class in future surveys.

• Cross reference with end of program data to examine specific cohorts that are more or less successful and isolate why.

Spring 2012 MBA Alumni Survey


Performance Measure


Analysis of Results



90% positive response to the overall TEDx event by Portland Community members

TEDx Survey (annual)

94% of respondents consistently rate the event as “Good” or “Brilliant” with the “Brilliant” response increasing steadily.

Exceeded our goal of 90% positive.

Continue to monitor overall satisfaction and ask specific questions regarding speakers, venue, etc. Add an incentive to raise the response rate to the survey (currently 22%) to ensure that these are valid results.

Internship Sponsors will rate 90% students at the “A” or “B” level for their overall preparation for an internship.

Intern Evaluation Form (each semester)

Meeting 90% goal. Increased satisfaction noted in Fall and Spring 13 internships

This data represents a summative review of overall satisfaction of internship sponsors with the preparation of students for internships in their organizations. This data is reviewed at the end of each semester to ensure we are meeting the overall goal. Time is also spent reviewing each individual student’s progress on the internship, as well as scores on detailed performance criteria to support individualized formative coaching sessions.

33 41 48

61 53 46

0102030405060708090

100

2011 2012 2013

% o

f Res

pons

es G

iven

Event Year

TEDx 2011-2013 Event Attendee Overall Rating

Good

Brilliant

100 90 91

100 100

05101520253035404550556065707580859095100

Fall 2011 Spring 2012 Summer 2012 Fall 2012 Spring 2013

Perc

ent

Semester

Intern Evaluation Form for Fall 2011 to Spring 2013 Percent of Students who received an A or B in relation to the Learning Objectives: Knowledge and Skill Applied to

Internship


Summary of graphical data in response to Criterion 3.8:

As demonstrated in the graphs and tables in the preceding pages, a broad range of student and stakeholder groups are surveyed regularly to ensure that the University and the School of Management take actions that ensure alignment is maintained between our mission, vision, values, and program outcomes. The following paragraphs will provide a summary of the preceding graphs, with the audience segment bolded for clarity and alignment to the graphs.

Concordia University regularly surveys its regional population, prospective students, current students, and other audiences to monitor perception of the University and its mission and core themes specifically. To date, this bi-annual telephone survey project has not isolated data for specific colleges or programs within the University, but instead has provided generic data for our use from the surrounding community and from students who inquire about our programs. The research study that yielded the first three charts in the Criterion 3.8 section (labeled “Institutional Brand Survey”) came from this University-wide survey. In future surveys, specific questions will be added to isolate responses relevant to the School of Management. Additionally, alumni will be added to the audience for this research survey, which has the potential to provide richer and more actionable data that the standard email based alumni survey that the University had been using in the past.

End of course assessments are used by Concordia University to assess current student responses to each course at the end of the semester. These assessments yield data that can be used by individual instructors, as well as the programs and School of Management as a whole, to improve instruction, curriculum, and overall student experience. A challenge that the School of Management has faced has been the response rate to these email-based end of course assessments. Historically, the School of Management has struggled to meet a minimum 30% response rate to these assessments, without using incentives like extra credit points that can interfere with accurate grading and assessment of student learning outcomes. In Fall 2012, the School of Management conducted an experiment, using a paper-based assessment with the same questions as the online assessment. A coordinator from the SOM attended each class session during the last 2 weeks of the semester to administer the survey. While this experiment yielded a much higher response rate, it was deemed too labor intensive to be sustainable. However, it did provide a venue for discussion with students about the importance of responding to end of course assessments, and helped to reinforce that the data from the assessments is important to the SOM and that students were being listened to and their feedback was valued. After this experiment, the SOM returned to the online end of course assessment, with an increase (from before the experiment) to a 46% response rate. We will continue to reinforce the importance of the assessments, reminding students and faculty when it is time to complete them, and will begin providing a summary to students of the changes made to courses based on their feedback. Another graph in Criterion 3.8 that demonstrates data from current students is the Undergrad Orientation Survey, which has provided consistent data for the past two years that athletics is an important interest on the part of our students, and reinforces our focus on undergraduate residence programs that leverage the interest in athletics. This data, and other data gathered through the 2012-13 BA undergraduate program review (Exhibits 24a and 24b), are being used to support the introduction of a Sports Management major to respond to the needs of the Portland fitness industry and the interests of our students. Additional data from current students is gathered in response to specific decisions, such as the choice of concentrations in the MBA program redesign (Exhibit 22 – MBA Survey Results).


Data from graduates is gathered through end of program surveys each semester. The data is reviewed by the SOM program directors and cross referenced with other data (from student outcome assessments and other sources) to make decisions regarding changes to programs, instructors, and courses. The graphs chosen for display in the preceding pages show that overall program satisfaction data is used as an overarching, summative data point. Note that Cohort 20 (which graduated in Spring 2012) reported a significantly lower satisfaction level than past cohorts. This caused a deeper analysis of the opened ended responses to the survey, as well as a review of end of course assessments, by the incoming Dean and MBA Program Director. The insights gathered from this review contributed to the redesign of the MBA that is being finalized in Spring 13, and will be introduced to students beginning in Fall 13. Also note that the response to this survey rose back to prior levels for Cohorts 6, 21, and 22, partially in response to multiple avenues opened up for student input and communication by the new Dean and Program Director.

The graph used to demonstrate focus on alumni input was created as an outcome of the 2012 alumni survey facilitated by Concordia’s Alumni Office. This survey has historically been conducted every 4 years, and in 2012 the School of Management partnered with the Alumni Office to facilitate a survey that included the BA in Business Administration students as a subset of the larger survey. The response rate to this survey was 9.7%. While the survey did provide some valuable insight into the perception of the alumni who responded, it did not provide adequate evidence that the degree earned at Concordia had been beneficial to the careers of our graduates, and with the low response rate and lack of data on graduating class, it was not possible to address this issue without more detailed data. The MBA alumni survey, however, was conducted directly by the School of Management, using contact information that had been maintained by the SOM. This survey yielded a 100% response rate and was able to provide focused data that supported the success of our students, specifically with applying the experience from their internship to their career after graduating. Future endeavors aimed at gathering information from alumni will be targeted on specific classes and cohorts within the SOM (similar to the MBA alumni survey) and will utilize telephone contact as well as electronic survey approaches to increase response rate and also provide richer qualitative data to support the quantitative data displayed in this report.

Data from the Portland community is gathered qualitatively through the relationships that faculty members maintain with their business industry networks. Additionally, surveys are used at the end of our annual TEDx event and after each internship to ensure that the needs of our local community for meaningful professional and personal development events (TEDx) and for career-ready graduates (internship survey) are being met. The TEDx event has been held at Concordia Portland, facilitated by SOM’s Dr. Michelle Jones, for the past three years. This event not only provides a learning and collaboration forum for the Portland community, but engages students, faculty, and staff members as volunteers to plan, design, develop, and host the event each year. The response to the event has been very positive, as evidenced by the survey data in the preceding pages. One area for improvement in this survey is the response rate, which has consistently been 22% for the past two years. Incentives to increase the survey response rate are being considered for the Spring 2014 event.

The internship sponsor feedback tracking represents an overall view of the perception of our undergraduate students for internships. This data is reviewed at the end of each semester to ensure we are meeting the overall goal. Time is also spent reviewing each individual student’s progress on the internship, as well as scores on detailed performance criteria to support individualized formative coaching sessions.


Standard 4 – Measurement and Analysis of Student Learning and Performance

Criterion 4.1 – Learning Outcomes Assessment Program Concordia University’s School of Management has developed and aligned the business curriculum (undergraduate and graduate) to a learning outcomes structure that includes The Concordia Mission (articulated through our three Core Themes), Concordia Values, School of Management Mission and Outcomes, Graduate and Undergraduate Outcomes, and Business Practice Area outcomes for all of the concentrations within the Graduate and Undergraduate programs. This structure is documented in Exhibit 12 - SOM Graduate & Undergraduate Outcomes Structure.

The outcomes documented in Exhibit 12 are embedded in the courses within each Graduate and Undergraduate program, providing an alignment between what is taught, what is measured, and evidence that supports the commitments we have made to the students and institution through our outcomes assessment program. The outcomes assessment program is further documented for tracking and monitoring in Concordia’s TracDat system. Access to the detailed assessment data in TracDat will be made available electronically during the site visit. The results of assessment activites are documented in Exhibit 13 – SOM Assessment Report.

Exhibit 10 - ACBSP Standards and SOM Measurement Methods list provides a detailed listing of the primary instruments deployed to support Concordia School of Management’s outcomes assessment program, and documents the location where the instruments and data are stored in the School of Management file sharing system. This includes surveys as well as direct assessment of learning outcomes through standardized testing. Additionally, Exhibit 11 ACBSP Calendar for 2012-13 details the process used within the School of Management for tracking completion of assessment tasks. This working document is used to direct the assessment activities at the beginning and end of each semester, ensuring that necessary surveys and exams are administered and results documented.

Figure 4.1 provides a summary of the internal, external, summative, and formative data that the School of Management gathers to support its learning outcomes program.

Figure 4.1 – Table for Student Learning Outcomes Assessment Data Degree Program Internal Data and

Information External Data and Information

BA in Business Administration

BA Orientation Survey BA Alumni Survey

End of Course Assessments ETS Major Field Test / Exit Exam Basic Critical Thinking Skills Test

(BCTST) – Pre and Post

End of Degree Evaluation - BA


Summative Data and Information

Formative Data and Information

ETS Major Field Test / Exit Exam BA Orientation Survey Basic Critical Thinking Skills Test

(BCTST) – Post Test End of Course Assessments

End of Degree Evaluation - BA and BA Alumni Survey

Basic Critical Thinking Skills Test (BCTST) – Pre Test

Degree Program Internal Data and Information

External Data and Information

Master in Business Administration (MBA)

End of Degree Assessments - MBA MBA Alumni Survey

End of Course Evaluations ETS Major Field Test / Exit Exam Basic Critical Thinking Skills Test

(BCTST) – Pre and Post

Summative Data and Information

Formative Data and Information

End of Degree Evaluation – MBA and MBA Alumni Survey

End of Course Assessments

ETS Major Field Test / Exit Exam Basic Critical Thinking Skills Test

(BCTST) – Post Test Basic Critical Thinking Skills Test (BCTST)

Criterion 4.2 – Reporting Results and Trends Exhibit 13 – SOM Assessment Report provides detailed documentation of the assessment results gathered to date. The data in this report is sorted in descending order (with the most recent results on each outcome listed first.) This report includes results from ETS Major Field Tests administered from 2009-2013, with data points for each semester when the tests were administered (8 data points so far). Undergraduate and graduate students take the ETS Major Field Test during their final semester with Concordia. This report was created in TracDat and will be available for more detailed analysis during the site visit. From the Exhibit 13 electronic file, you can click on embedded jpg files to view the findings graphically. Note on the graphics linked to Exhibit 13: This exhibit demonstrates the specific outcomes assessment method for each outcome, which includes ETS Major Field Tests and the use of the BCTST Critical Thinking Skills test at the undergraduate and graduate level. The Concordia SOM student scores (blue line labeled Avg) are plotted against national averages calculated by the testing agency (red line labeled N. Avg).

Also included are the scores from BCTST exams from 2011, also with data points for pre-tests each semester. The BCTST is given in the first semester (pre-test) and (beginning in 2013) graduating students’ final semester. The first group of undergraduate students to take the BCTST post-test was the graduating class in Spring of 2013, based on Fall 2011 pre-tests.


Criterion 4.3 – Selection and Use of Comparative Information See Figure 4.2 for a summary of assessment results and Exhibit 13 – SOM Assessment Report – for a full report on all assessment activities, aligned to program outcomes and ACBSP standards.

Criterion 4.4 – Continuous Process Improvement The School of Management has developed a process of reviewing formative data in Faculty meetings at the end of each semester, with an in-depth review at each annual Fall Faculty Retreat. Exhibit 14 - Student Outcomes Data Meeting PPTs provides detailed documentation of the data presented and discussions conducted in response to the data at two annual Fall Retreats (2011 and 2012) and a review in January, 2013. See Exhibit 13 – SOM Assessment Report for documentation of the performance measures, results, analysis of results, actions taken, and next steps. The next scheduled formal review of outcomes data will be at the Fall 2013 Faculty Retreat, scheduled for August 22, 2013. Figure 4.2 highlights examples of assessment efforts that have resulted in specific improvements.

Figure 4.2 – Example Improvements based on Assessment Data - BA Performance Measure / Instrument

Analysis of Results

Action Taken (Improvement)

Graph of Resulting Trends

BA Students will score no more than 10 points below the national average in the Finance category. ETS Major Field Test

Current students = 43 National norm = 42.8 With modest improvements in subsequent terms, students have exceeded national norms for the first time in Spring 13.

Additional focused content was reinforced in the course to improve learning and formatively tested with embedded questions. Revise goal to “Students will meet or exceed national average” now that content has been enhanced.


Figure 4.2 – Example Improvements based on Assessment Data – BA (continued) Accounting: Stable scores, slightly above national norms since Spring 2010

Economics: Very close to national norms since Spring 2011. Exceeded national norm by a small margin for the first time in Spring 2013

Finance: Consistently right at or below national norms since Spring 2011. Exceeded national norm by a very small margin for first time in Spring 2013.

Figure 4.2 – Example Improvements based on Assessment Data – BA (continued)


Information Systems: Consistently below national norms with a few exceptions. Exceeded national norm by more than 5 points for the first time in Spring 2013.

International Issues: Had been consistently below national norm until Spring 2012 and Fall 2012 when we exceeded by more than 5 points. Back down to 3 points over in Spring 2013.

Legal and Social Environment: Consistently right at the national norms since Spring 2011. Exceeded national norm by more than 5 points in Spring 2013.

Figure 4.2 – Example Improvements based on Assessment Data – BA (continued)


Management: Consistently right at, above, or below national norms since Spring 2010. Exceeded national norm by a very small margin in Spring 2013.

Marketing: Consistently right at, above, or below national norms since Spring 2010. Exceeded national norm by a very small margin in Spring 2013.


Figure 4.2 – Example Improvements based on Assessment Data – BA (continued) Quantitative Business Analysis: Just above national norms since Fall 2011.

BA outcome review summary: As noted previously, the outcomes data for BA students has remained fairly flat over the past few years, indicating that the program outcomes (as currently designed) were being met. What this data does not tell us, however, was whether we were measuring the right outcomes. That question, coupled with data from student surveys, end of course assessments, and conversations with students, faculty, and business partners, led to the initiation of a thorough program review which combined a review of quantitative data with a series of qualitative focus group and interview sessions across stakeholder groups, a review of the competitive landscape, and an analysis of the current research on undergraduate student and industry needs. That review, called out in the SOM Strategic Plan, was conducted in AY 2012-13 and is detailed in Exhibits 24a and 24b. In addition to initiating a redesign of the curriculum, the results of the study will require a re-establishment of metrics to align to revised program outcomes and a review of the assessment instrument being used to measure student outcomes.


Figure 4.2 – Example Improvements based on Assessment Data - MBA Performance Measure / Instrument

Analysis of Results

Action Taken (Improvement) Graph of Resulting Trends

MBA Bend students will score at least 10 points higher than the national average overall. ETS Major Field Test

Scores are consistently exceeding standard. Fewer MBA programs in Bend, yielding ability to recruit higher achieving students than in Portland.

Take advantage of this trend by blending Bend and Portland students in online electives for learning transfer.

MBA Portland students will meet or exceed the national average overall. ETS Major Field Test

Students have continued to score approx. 10 points below national norm.

Look at individual topic scores to isolate the problem. Closer examination shows that Management and Accounting are the areas where the most improvement is needed, and Finance is a topic that has been improved. See subsequent topic data in the next 3 charts for analysis.


Figure 4.2 – Example Improvements based on Assessment Data – MBA (continued) Performance Measure / Instrument

Results

Analysis of Results Action Taken (Improvement)

MBA Portland students will meet or exceed the national average overall. ETS Major Field Test

Finance: 5-7 points below national norm in past 3 cohorts, with an increase of 9 points (3 points above national norm) in Spring 13.

Assigned full-time professor to teach MBA 508 (Finance) in Fall 2012 (Ron Miolla). Professor reviewed and improved content in course, resulting in an increase in student achievement for exiting students in Spring 2013. This improvement in student success in the Finance concentration has led to decisions related to content review in Management and Accounting:

Ensure that the assessment tool is aligned to ACBSP standards and consider adopting Peregrine’s assessment program as new MBA curriculum is more clearly aligned to ACBSP standards.

Management: 5-10 points below national norm in past 4 cohorts.

Accounting: 5 points below national norm in past 4 cohorts.


Figure 4.2 – Example Improvements based on Assessment Data – MBA (continued) Performance Measure / Instrument

Analysis of Results

Graph of Resulting Trends

MBA Students will report an average of 8 out of 10 on satisfaction on all program components MBA End of Degree Survey

Cohort 20 scores lower than previous cohorts. All cohorts (Bend and Portland) reported low satisfaction with “use information technology in an advanced way.”

This and other data from Cohort 20 indicates issues with satisfaction that will need to be monitored in future cohorts.

Action Taken (Improvement) Changes were made to MBA 501 course to enhance technology integration, including a new faculty member with more expertise with technology and changes to curriculum. Changes have resulted in an increase in the info tech score to 7.5 (from a low of 4.5). Continue to look for ways to emphasize integration of information technology and monitor this trend.

As previously detailed, both the BA and MBA programs were either revised (MBA) or entered into an intensive review process (BA) during our self-study year. As an outcome of those revisions, outcomes will be remapped to assessment instruments for the coming academic year. Concordia is currently reviewing the fit of the ETS assessment instruments as well as Peregrine’s instruments to ensure that the metrics for both programs are well aligned to the re-designed outcomes, as well as ACBSP standards.

Concordia University ACBSP Self-Study Standards 3 and 4 ......Portland Business Community Exposure to best practices in community well-being, sustainability, and social entrepreneurship.

Documents