The Competitive Edge: Web-based Assessment of Learning Joyce A. Scott, Trustee Richard C. Thompson, Dean Jones International University © Jones International.

The Competitive Edge: Web-based Assessment of Learning

Joyce A. Scott, Trustee

Richard C. Thompson, Dean

Jones International University

© Jones International University®, Ltd. 2012

Copyright Jones International University, 2012.


This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

Jones International University

• Founded in 1993• First totally online university• Accredited in 1999 by the

North Central Association’s HLC

• “…the success of our students and the viable, global democratization of education are our urgent concerns.” (excerpted from JIU Vision)


Student Demographics1

• 74% work full time and 11% are not employed 2

• 67% are age 30 or older• 57% are women• Diversity

– 36% African American– 3.7 % Hispanic– 32% White

• 49% earned $40,000+ annually prior to enrollment

1 Data as of January 1, 20122 85% of respondents provided employment data


Adult Learning• “…is subsidiary and supplemental to a primary

productive role in society”—Verner (1965)

• Students need to know relevance, utility, application

• Adult students need

– To be self-directing

– Task-centered learning – Time and conditions for reflection– Feedback to guide learning, correct mistakes,

improve learning, model professional practice


Assessment

• Involvement in Learning (1984)– Set standards of learning– Publish standards so students can internalize – Give feedback to guide/redirect effort

• Integrity in the College Curriculum (1985)– Faculty should examine learning outcomes– Evaluate efficacy of students, faculty, programs

• Levels of assessment: From the student to the institution (2005)


JIU Course Design Model


• Philosophical outlook

– Adult learner focus– Consistent student experience– Centralized curriculum development

• Key Elements– Assigned responsibilities– Specific format (look/feel, pedagogical construct, etc.)– Assessment built into courses

JIU Assessment InfrastructureInstitutional Learning Outcomes→Workplace Competencies

– Communicate effectively using a variety of methods and disciplines

↓

Program Learning Outcomes (MBA)– Apply selected principles of business communication to a variety of

management challenges

↓

Course Learning Outcomes– Communicate effectively/persuasively both orally and in written form, in

a variety of multi-cultural contexts. [BC607 Leading from a Global Perspective]

Institutional Learning Outcomes (ILO)

Each graduate will have demonstrated the ability to: – think and act like a global citizen with personal integrity, respect

for diversity, and ethical behavior

– think and act like leaders and supportive teammates

– adapt and apply theories to solve “real world” problems using innovative and creative solutions

– communicate effectively using a variety of methods and disciplines

Institutional Learning Outcomes-2 – communicate effectively using a variety of methods and

disciplines

– think analytically, critically, and systemically to prepare for a life of learning and accomplishment

– leverage information and technology to amplify personal and organizational achievement, and promote information literacy

– design a professional development strategy to formulate a career pathway leading to future success

Workplace Competencies (WC)


• Convert Institutional Outcomes to measurable indicators.

• Workplace Competencies use concepts from specific elements of the ILOs and translate into practical indicators:

- Completeness - Critical Thinking- Timeliness - Synthesis- Originality - Clarity- Feedback Incorporated

Program Learning Outcomes (PLO)


• At JIU, PLOs are anchored whenever possible in:

– Standards from regulatory bodies, or

– Standards of relevant professional associations, or

– When all else fails, standards are developed by a Program Advisory Committee convened by the university

PLOs


• Example--outcomes for MBA-Project Management :

– Knowledge• Examine different ethical reasoning concepts, theories, and issues

as they apply to the conduct of business in global and e-commerce environments.

– Skills• Assess budgeting, cash management, credit administration,

investment analysis, borrowing funds and financial forecasting tools for effective business decision making.

– Project Management Proficiencies• Develop timelines, conduct evaluations, and measure overall

strategic impacts for projects.

Course Learning Objectives (CLO)


• CLOs:– developed by faculty, reviewed by curriculum committees– built for each course within the context of a curriculum map.– connect each course to the Program Learning Objectives.

• CLOs naturally delve into details of learning outcomes within the course

Competitive Edge or Advantage

• Resources and Capabilities create Distinct Competencies

• Lead to unique value creation

• Assessment system creates value for both the students and the university– For value added competitive advantage models see Porter’s collective work on Competitive

Advantage or Wernerfelt’s Resource-Based View (RBV).

Assessment & The Competitive Edge – Student View

• Up-front knowledge of learning expectations

• Directly measured against those expectations

• Student Success Data show progress, current and progressive

• Individual • Compared with peers

Individual Feedback


Student view of performance

in a specific course


Individual StudentComparative Growth

Student Summary Feedback


Student view of performance across all course outcomes and Workplace Competencies

The Learner’s Real-World Portfolio

– Final course assessments– Real-world experience in the profession*– Professors’ summative evaluations– PSP report or document*– External sponsor’s evaluation*– Projects*– Resume and other job-related documents

* For students engaged in project-based courses or final professional synthesizing projects


JIU’s Competitive Edge

• Student Success Data and Student End of Course Surveys are aggregated at:– Course Level– Specialization and Program Level

• Combination of two assessments provides a rich view of the performance of courses and programs and is used by:– Faculty during course revision– Curriculum Committees– Program Review Committees

Bi-Annual Assessment Report


School of Business: Student Achievement of Course Learning Objectives and Workplace Competencies by Course

Legend: Exemplary: 91%-100% of Course Learning Objectives are assessed as Proficient or Advanced Average: 75%-90% of Course Learning Objectives are assessed as Proficient or Advanced Needs Review: 0% - 74% of Course Learning Objectives are assessed as Proficient or Advanced


Student Success Data:Aggregate Achievement of Workplace Competencies and Course

Learning Objectives at the Program Level

Student End of Course Survey Results

Student End of Course Survey 2011MBA Program and Project Management Specialization

Scale is 1 to 5

Course Quality QuestionsMBA Dec'11

MBA 2011 Ave. MBA571 MBA572 MBA573 MBA574

Project Management Specialization

This course was well designed. 4.10 4.26 4.40 4.53 4.38 4.55 4.47This course was appropriate for my profession. 4.29 4.42 4.68 4.77 4.53 4.63 4.65Learning objectives were clearly stated at the beginning of this course. 4.45 4.49 4.57 4.63 4.34 4.70 4.56My work in this course was high quality. 4.28 4.34 4.51 4.57 4.47 4.63 4.55I learned what I was expected to learn in this course. 4.24 4.34 4.46 4.59 4.44 4.63 4.53This course allowed me to be an active learner. 4.26 4.39 4.57 4.63 4.34 4.52 4.52This course allowed enough time for me to be successful. 4.17 4.17 4.09 4.48 4.29 4.45 4.33This course's materials and readings aided my learning. 4.23 4.30 4.47 4.43 4.47 4.63 4.50

MBA571 The Project Management Framework MBA572 Planning Projects to Manage OutcomesMBA573 Project Controls and Tracking Project ProgressMBA574 Strategic Thinking and Project Management

Next Phase• Implement a packaged assessment system (such as

TaskStream or LiveText) to– Integrate/organize assessment data from current systems– Provide structured portfolios for students– Organize/aggregate student work for accreditation reporting; e.g.

HLC, NCATE

• Develop a Business Intelligence system to– Incorporate packets of data from disconnected systems– Allow for rich university evaluation of assessment data by

student demographics, financial performance data, and even national databases

References

• Association of American Colleges (AAC). (1985). Integrity in the college curriculum: A report to the academic community. Washington, D C.

• Brookfield, S. D. (2006). The skillful teacher: On technique, trust and responsiveness in the classroom, (2nd ed.) San Francisco: Jossey-Bass.

• Knowles, M., Holton, E., & Swanson, R. (1998). The adult learner: The definitive classic in adult education and human resource development (5th ed.). Houston: Gulf Publishing Co.

• Miller, R. & Leskes, A. (2005). Levels of assessment: From the student to the institution. Washington, DC: AACU.

• U.S. Department of Education. (1984). Involvement in learning: Realizing the potential of American higher education. Washington, D.C.: Author

• Verner, C. (1965). Programming. Review of educational research, 35(3), 177-184.• Wernerfelt, B. (1984). A Resource-based View of the Firm. Strategic Management

Journal, 5(2), 171–180.

For further information about this presentation:

www.cyberschools.com

For further information about Jones International University (JIU):

www.jiu.edu


http://www.cyberschools.com/

http://www.jiu.edu/

The Competitive Edge: Web-based Assessment of Learning Joyce A. Scott, Trustee Richard C. Thompson, Dean Jones International University © Jones International.

Documents

adult learning

online university

set standards of learning

global perspective slide

application adult students

efficacy of students

assessment involvement

written permission