“Linking M A Program and ourse Learning Outcomes to ... · MBA Program Workshop “Linking M A Program and ourse Learning Outcomes to QFEmirates Level 9 Descriptors” Thursday

MBA Program Workshop

“Linking MBA Program and Course Learning Outcomes to QFEmirates Level 9 Descriptors”

Thursday 28th May, 2015

0930 – 1230

Hosted by the University of Sharjah

1

Facilitators

Professor James Jordan-Wagner Dean, W Frank Barton School of Business, Wichita State

University, USA

Professor Mostafa Sarhan Dean, College of Business Administration, Savannah State

University, USA

Dr Wil Williams Commission for Academic Accreditation

2

Purpose

To improve the writing of MBA program (PLOs) and course learning outcomes (CLOs) and the design of assessment regimes and instruments that should be aligned to QFEmirates Level 9 Descriptors.

3

Learning Outcomes

Participants in this workshop will gain: – an overview of key issues that the CAA have

identified in MBA program initial, full and renewal accreditation events;

– an in-depth understanding of the relationship between QFEmirates and the CAA’s Standards;

– a critical appreciation of writing MBA PLOs and CLOs at an appropriate level;

– an understanding of how to design MBA assessment regimes appropriate for the level of study; and

– a critical appreciation of MBA assessment instruments that are set at an appropriate level.

4 H

Agenda

Time Activity

0900 - 0930 Registration & coffee

0930 - 0945 Welcome & introduction

0945 - 1015 The nature of the MBA in the UAE - Issues with Initial and Renewal of MBA Accreditation

1015 - 1045 QFEmirates Level 9 and the CAA’s Standards

1045 - 1115 Guide to writing PLOs and CLOs

1115 - 1130 Coffee

1130 - 1200 Linking PLOs and CLOs to assessment

1200 - 1230 Plenary

1230 Depart

5

The Nature of the MBA in the UAE - Issues with Initial and Renewal of MBA Accreditation

6

7

The Nature of the MBA in the UAE - Issues with Initial and Renewal of MBA

Accreditation

• 32 MBA programs in the UAE • Earliest: 2001 • Latest: 2015 • Number of students - Fall 2014: 6,403 • Initial Accreditation • Renewal of Accreditation • CAA Standards (2011) • QFEmirates (2012)

8

9

MBA: Private Institutions

10

MBA: Private Institutions

11

14

4

2

2

11

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

Fall 2010 Fall 2011 Fall 2012 Fall 2013 Fall 2014

MBA Total Number of Students By Gender

Female

Male

12

0 1,000 2,000 3,000 4,000 5,000 6,000 7,000

Fall 2010

Fall 2011

Fall 2012

Fall 2013

Fall 2014

MBA Total Number of Students By Nationality

UAE

Expat


Accreditation

Issues –

• Institutional Planning (2.2): Too few programs evidence ‘closing the loop’

• Credit-bearing Program (3.1):

– Lack of market intelligence and analysis (3.1.1)

– Poor alignment of PLOs, CLOs and assessment to QFEmirates Level 9 Descriptors (3.1.3)

– Full-time programs being completed within 12 months – Too few contact and learning hours (3.1.5)

13


Accreditation

Issues (Cont.) –

• The Curricula (3.2)

– Structured more like specialist MSc programs (3.2.1)

• Academic Courses (3.3)

– Lack of evidence of student active engagement (3.3.2)


– Cont.…

14

Issues (Cont.) –

• Academic Courses (Cont.) (3.3) – Use of inappropriate assessment tools, e.g., quizzes, short (very)

essay type questions and ‘participation’ (attendance) (3.3.4)

– Syllabi (3.3.5; Stipulation 5)

• Poor rationale for course selection (2)

• The lack of grading rubrics/ guides

• Insufficient information on course topics (N.B. Rarely explicitly linked to CLOs and assessment) and assessment;

• Out of date texts; and

• Lack of references to journals (5)

– Cont.…

15


Accreditation

Issues (Cont.) –

• Academic Courses (Cont.) (3.3) – Condensed programs/ courses: too little time allowed for

preparation, reflection and analysis (3.3.6; also see 3.1.5)

– Lack of evidence of the use of adequate plagiarism prevention safeguards (3.6.8)

– Insufficient infrastructure to support blended learning provision (3.3.9)

• Graduate Programs: – Lack of research training (3.7.1)

– Lack of evidence of sufficient rigor, and courses being distinctly more challenging than undergraduate courses in the same discipline (3.7.2)

16


Accreditation


Accreditation

Issues (Cont.) –

• Course Delivery:


– Course files stored and presented in poor order (3.8.3)

• Class sizes – too large (3.9)

• Program Effectiveness - Too few programs evidence ‘closing the loop’ (3.10)

17


Accreditation

Issues (Cont.) –

• Faculty

– Lack of evidence of professional development - research and scholarship (4.8.1)

– High workloads (4.9)

– Too many part-time faculty (4.10)

• Student Recruitment:

– Students lack business and management experience (5.3.1.1)

– High number of conditional entrants (5.3.1.8)

18


Accreditation

Issues (Cont.) –

• Public Disclosure – Poor presentation of accreditation (self studies) documentation (9.2; also see 3.8.3)

• Research and scholarship – weak faculty research outputs (10)

19

QFEmirates Level 9 and the CAA’s Standards

20


QFEmirates Strands

21

1 2 3 4 5 6 7 8 9

10

Knowledge

Leve

ls

Skills Competence

22


• QFEmirates Level 9 – Annexure C: Grid of Level Descriptors

– Annexure D: Summary of Level Descriptors

– Annexure G: Appendix 18 – Generic Qualification Profile 9.1 (Master)

– Annexure F: Employability Indicators of QFEmirates Levels

23

Level Knowledge Skill

Competence

Autonomy &

responsibility

Role in

context

Self-

development

9 comprehensive, highly specialized knowledge

in a field of work,

discipline and/or professional practice,

and at the interface

between different fields,

including frontier

concepts and recent

developments

advanced knowledge of

applicable research principles and methods

critical awareness of

knowledge issues, as the

basis for original

thinking; encompassing appropriate processes of

enquiry and current

processes of knowledge production

detailed body of

knowledge of recent

developments in a field

of work, and/or discipline

advanced skills required in research,

analysis, evaluation

and/ or innovation of complex ideas,

information, concepts

and/or activities

skills to develop new knowledge and

procedures and to

integrate knowledge from different fields

using highly developed

cognitive and creative

skills and intellectual

independence to the

field of work or discipline

advanced problem-solving skills to analyze

highly complex issues

with incomplete data

and develop innovative

solutions and proposals

relevant to an academic/ professional

field, field of work or

discipline

planning skills to

develop and execute a major project or

comparable activities

(that includes a significant range of

variables and

complexity) with appropriately selected

research methodologies

producing sound conclusions

highly developed specialist

communication and information technology

skills to present,

explain and/or critique highly complex matters

can function autonomously and/

or take responsibility

for managing professional

practices, work,

processes or systems,

or learning

contexts that are

highly complex, unpredictable and

unfamiliar, and

require new strategic approaches

and/or intervention

or conceptual

abstract solutions

can account for high

level governance of

processes and

systems can analyze and

reflect on

sociocultural

norms and

relationships

and act to build and transform

them

can initiate and manage

professional

activities that may include a

highly complex

environment

can take

responsibility for leading the

strategic

performance and development of

professional

teams and self

can self-evaluate and take

responsibility for

contributing to professional

knowledge and

practice including

unfamiliar learning

contexts

can develop and

implement further

learning consistently and

sensitively can

consistently and

sensitively

manage highly complex ethical

issues leading to

informed, fair and valid decisions

QFEmirates Level 9 and the Standards

• QFEmirates Level 9 - Annexure C: Grid of Level Descriptors

24 H


• QFEmirates Level 9 - Annexure C: Grid of Level Descriptors • Strand: Knowledge

– comprehensive, highly specialized knowledge in a field of work, discipline and/or professional practice, and at the interface between different fields, including frontier concepts and recent developments

– advanced knowledge of applicable research principles and methods

– critical awareness of knowledge issues, as the basis for original thinking; encompassing appropriate processes of enquiry and current processes of knowledge production

– detailed body of knowledge of recent developments in a field of work, and/or discipline

25 H Grid



• Strand: Skills

– advanced skills required in research, analysis, evaluation and/ or innovation of complex ideas, information, concepts and/or activities

– skills to develop new knowledge and procedures and to integrate knowledge from different fields using highly developed cognitive and creative skills and intellectual independence to the field of work or discipline

– advanced problem-solving skills to analyze highly complex issues with incomplete data and develop innovative solutions and proposals relevant to an academic/ professional field, field of work or discipline

– planning skills to develop and execute a major project or comparable activities (that includes a significant range of variables and complexity) with appropriately selected research methodologies producing sound conclusions

– highly developed specialist communication and information technology skills to present, explain and/or critique highly complex matters

26 H Grid



• Strand: Competencies - Autonomy and Responsibility

– can function autonomously and/ or take responsibility for managing professional practices, work, processes or systems, or learning contexts that are highly complex, unpredictable and unfamiliar, and require new strategic approaches and/or intervention or conceptual abstract solutions

– can account for high level governance of processes and systems

– can analyze and reflect on sociocultural norms and relationships and act

27 H Grid



• Strand: Competencies – Role in Context

– can initiate and manage professional activities that may include a highly complex environment

– can take responsibility for leading the strategic performance and development of professional teams and self

28 H Grid



• Strand: Competencies – Self-Development

– can self-evaluate and take responsibility for contributing to professional knowledge and practice including unfamiliar learning contexts

– can develop and implement further learning consistently and sensitively can consistently and sensitively

– manage highly complex ethical issues leading to informed, fair and valid decisions

29 H Grid


• Standards explicitly linked to QFEmirates: – 3.1.3

– 3.3.3

– 3.8.2

30


• Standards linked to QFEmirates: – 3.1.3 Each academic program has well-articulated program goals

and outcomes (i.e. learning outcomes at the level of the program); the outcomes are derived from the goals and are defined in measurable terms; all program goals and outcomes are accurately described in appropriate publications. Goals and outcomes are consistent with the level of qualification awarded as defined in the UAE Qualifications Framework.

– 3.3.3 provides learning outcomes for each course that are consistent with current international norms in the discipline and the level of the course and the qualification awarded; and align with, and demonstrably meet, the program outcomes of each program in which the course appears

– 3.8.2 ensures that the academic assessment of students is fair, accurate, aligned with learning outcomes and program goals, and is undertaken at an appropriate level

31 H


• Exercise: Card Sort

– Which of the cards are acceptable PLOs and CLOs • Upon successful completion of this program/ course, the

graduate will be able to: – Understand social, professional and/or ethical responsibilities in business

environment

– Communicate business issues effectively

– Critically examine the role of management accounting information and related ethical issues in planning, organizing, and controlling activities of a firm

– Implement a strategic plan that takes into account the key functional resources of a firm

– Develop advanced skills in business research application, analysis, and evaluation as well as the ability to innovate new ideas, concepts and business practice

32 E




graduate will be able to:

–Understand social, professional and/or ethical responsibilities in business environment

33 E





environment

–Communicate business issues effectively

34 E





environment



35 E





environment




36 E




graduate will be able to: – Understand social, professional and/or ethical responsibilities in business environment




–Develop advanced skills in business research application, analysis, and evaluation as well as the ability to innovate new ideas, concepts and business practice

37 E

Guide to writing PLOs and CLOs

38

Guide to Writing PLOs and CLOs

Definition of a learning outcome

“Learning outcomes are statements of what a student is expected to know, understand and/or be able to demonstrate after completion of a process of learning.”

(Directorate-General for Education, 2015, p.47)

39


• Aims, goals, objectives and learning outcomes

– Applicable to both programs and courses (module)

– An aim is a broad general statement of the teaching intention

– Goals are the broad educational expectations for each degree program (AACSB)

– An objective is a specific statement about the intended result of teaching (Instructor perspective)

– An outcome is a specific statement about the achieved result of teaching (Student perspective)

40


• Objectives and outcomes:

– Objectives – intended results

– Outcomes – achieved results (what you will measure)

41

“[A] general lack of agreement as to the format of objectives is a complication, and justifies the abandonment of the use of the term ‘objective’ in the description of modules or programmes.” (Moon, 2002)


42

Program - Mission/ Aim - Goals - Outcomes

Course - Mission/ Aim - Outcomes


• Learning outcomes

– A student-centered approach

– Closely linked to the competency approach to learning

43


Why use learning outcomes? Because they:

• help students learn more effectively through transparency;

• help instructors to design their teaching and learning approach and materials;

• help instructors more precisely to tell their colleagues and students what a particular activity is designed to achieve; and

• ensure that appropriate assessment approaches and instruments are employed.

(Source: Adapted from - SOAS, 2015)

44


45

LO

Simple

Measurable Achievable

Realistic

Timely

“On completion of this program/course/module the student should be able to …”

Action Verb

Scope

Guidance


• To achieve the required level focus on :

– The verbs used to describe the learning outcome;

– The scope of the task and quality of the guidance given to the students; and

– Quality of guidance: How much guidance the students will be given by the instructors and how much they are capable of doing things independently.

46


47

1 per LO

Limit the number

Action Verb

LO Minimum standard to pass


• Program Learning Outcomes (PLOs):

– An institution must specify PLOs for each separate degree program and concentrations within a degree

– PLOs will tend to be broader in nature

– PLOS assessed during, after or not at all

– Some PLOs maybe overarching

– Map PLOs to courses (and even CLOs)

48


• Program Learning Outcomes (PLOs):

– An institution must specify PLOs for each separate degree program and concentrations within a degree

– PLOs will tend to be broader in nature

– PLOS assessed during, after or not at all

– Some PLOs maybe overarching

– Map PLOs to courses (and even CLOs)

49


• Course Learning Outcomes (CLOs):

– Each course must specify CLOs

– They are specific

– Fulfil the requirements of one or more of the program learning outcomes;

– The CLOs should be mapped to the PLOs to ensure that the PLOS are met

50


• Course Learning Outcomes (CLOs):

– Each course must specify CLOs

– They are specific

– Fulfil the requirements of one or more of the program learning outcomes;

– The CLOs should be mapped to the PLOs to ensure that the PLOS are met

51


• A useful construct to link QFEmirates qualification levels to PLOs and CLOs is Bloom’s Taxonomy

• Bloom’s original taxonomy (1956) has three parts (or domains): 1. Cognitive – knowing

2. Affective – thinking

3. Behavioral (psychomotor)

– able to do

52


• A useful construct to link QFEmirates qualification levels to PLOs and CLOs is Bloom’s Taxonomy

• Bloom’s original taxonomy (1956) has three parts (or domains): 1. Cognitive – knowing

2. Affective – thinking

3. Behavioral (psychomotor)

– able to do

53


Bloom’s Taxonomy - Cognitive

1. Knowledge

2. Comprehension

3. Application

4. Analysis

5. Synthesis

6. Evaluation

Leve

ls o

f K

no

win

g

54


• Action Verbs: – Analysis

• Analyze, appraise, arrange, break down, calculate, categorize, classify, compare, connect, contrast, criticize, debate, deduce, determine, differentiate, discriminate, distinguish, divide, examine, experiment, identify, illustrate, infer, inspect, investigate, order, outline, point out, question, relate, separate, sub-divide, test.

– Synthesis • Argue, arrange, assemble, categorize, collect, combine, compile, compose,

construct, create, design, develop, devise, establish, explain, formulate, generalize, generate, integrate, invent, make, manage, modify, organize, originate, plan, prepare, propose, rearrange, reconstruct, relate, reorganize, revise, rewrite, set up, summarize.

– Evaluation • Appraise, ascertain, argue, assess, attach, choose, compare, conclude,

contrast, convince, criticize, decide, defend, discriminate, explain, evaluate, grade, interpret, judge, justify, measure, predict, rate, recommend, relate, resolve, revise, score, summarize, support, validate, value.

55 H


Table 3.5 Checklist for writing learning outcomes Have I focused on outcomes not processes, i.e. have I focused on what

the students are able to demonstrate rather than on what I have done in my teaching?

Have I begun each outcome with an active verb? Have I used only one active verb per learning outcome? Have I avoided terms like know, understand, learn, be familiar with, be

exposed to, be acquainted with, and be aware of? Are my outcomes observable and measurable? Are my outcomes capable of being assessed? Have I included learning outcomes across the range of levels of Bloom’s

Taxonomy? Do all the outcomes fit within the aims and content of the module? Have I the recommended number of outcomes (maximum of nine per

module)? Is it realistic to achieve the learning outcomes (Source: Kennedy, 2006 )

56 H


• Exercise 2: Grid to Action Verbs

– Examine QFEmirates : Annexure C: Grid of Level Descriptors

Identify key action verbs in the Level 9 Descriptors that relate to Bloom’s Levels:

• 4. Analysis

• 5. Synthesis

• 6. Evaluation

57 E

Linking PLOs and CLOs to Assessment

58


• Assessment

– Formative: assessment for learning

– Summative: summarizes student learning at a point in time

– Continuous assessment

– Grading rubrics

– Must be related to PLOs, CLOs, course content

– Must be set at the correct level, i.e., QFEmirates Level 9

59


• Assessment

– Formative: assessment for learning

– Summative: summarizes student learning at a point in time

– Continuous assessment

– Grading rubrics

– Must be related to PLOs, CLOs, course content

– Must be set at the correct level, i.e., QFEmirates Level 9

60


• Linking learning outcomes, teaching and learning activities and assessment:

1. Clearly define the CLOs

2. Select appropriate teaching and learning methods

3. Design an assessment regime that is appropriate to test the CLOs

4. Assess the learning outcomes and check to see how well they match with what was intended

5. Evaluate the research instruments in terms of the program

Source: Adapted from - SOAS, 2015)

61


• Measuring PLOs and CLOs

– Assessment is the only measure?

– Direct measures, e.g. examination and coursework results

– Indirect measures e.g., surveys of employers, comparison with peer institutions, surveys of past graduates, retention rates, analyses of curricula, etc.

62


• Exercise 3: Assessment Questions

– Using the action verbs presented devise examination or coursework questions for a MBA course (of your choice) that test a student’s ability to (one question per level):

• Analyze

• Synthesize

• Evaluate

63 E

Plenary

64

Aligning PLOs, CLOs and Assessment

Source: Adapted from Moon

Program Aim/s QFEmirates Level Descriptors

Course Learning Outcomes (CLOs)

Program Learning Outcomes PLOs)

Development of assessment method to criteria (criteria implied by test

achievement of assessment criteria)

Write threshold assessment criteria (criteria implied by test achievement

of assessment criteria learning outcomes)

Course Delivery Approach Feedback loop to program and course development

Course Review

65

Aligning PLOs, CLOs and Assessment

Source: Adapted from Moon

Program Aim/s QFEmirates Level Descriptors

Course Learning Outcomes (CLOs)

Program Learning Outcomes PLOs)

Development of assessment method to criteria (criteria implied by test

achievement of assessment criteria)

Write threshold assessment criteria (criteria implied by test achievement

of assessment criteria learning outcomes)

Course Delivery Approach Feedback loop to program and course development

Course Review

66

PROCESS TO DEMONSTRATE PROGRAM ALIGNMENT WITH QFEMIRATES

67

Next CAA Program

Accreditation Review

Report no changes

CAA Substantive Change

Submission

Report changes

Institutional/College Program

Review Committee

Oversight and

guidance from

Institutional

QA Office

Negative alignment and need

for curriculum revision

Positive alignment with

appropriate QFEmirates level

Program team to modify

curriculum

Outcomes Matrix Map against

QF level descriptors

Plenary

Examples of good practice

68

Alignment is more than a paper exercise

69

Thank you

[email protected]

70

71

Thanks to Prof. Ian Cumbus in the development of this material

“Linking M A Program and ourse Learning Outcomes to ... · MBA Program Workshop “Linking M A Program and ourse Learning Outcomes to QFEmirates Level 9 Descriptors” Thursday

Documents