Developing a Rubric as One Measurement Tool in the Assessment Process for Academic Advising Sharon A. Aiken-Wisniewski, Ph.D. University of Utah NACADA.

Developing a Rubric as One Measurement Tool

in the Assessment Process for Academic

Advising

Sharon A. Aiken-Wisniewski, Ph.D.University of Utah

NACADA Executive OfficeKansas State University

2323 Anderson Ave, Suite 225Manhattan, KS  66502-2912

Phone: (785) 532-5717   Fax: (785) 532-7732

e-mail: [email protected]

© 2012 National Academic Advising Association

The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association.

The Global Community for Academic Advising

An Activity

The Global Community for Academic Advising

The Global Community for Academic Advising

Participant Learning Outcomes

Behavioral

•Describe a rubric.

•Develop levels of performance for a rubric.

•Develop criterion points for a rubric.

Cognitive

•Understand the purpose of using a rubric for measurement.

•Explain the use of descriptors when considering a level of performance for a criterion point.

Affective

•Values reflection within the process of developing a rubric.

A rubric is a scoring scale utilized to measure a student’s or

advisor’s performance against a predetermined set of criteria

Assessment of Academic Advising InstituteAssessment of Academic Advising Institute

The Global Community for Academic Advising

A Rubric . . .• Divides a desired objective or outcome into

measurable pieces• Identifies what are acceptable and unacceptable

degrees of performance for a specific outcome• The component parts are called

– criterion points and – degrees of performance

• Descriptors are the intersection of criterion points and degrees of performance that assist with scoring

Rubrics typically have at least two criterion points

andtwo levels of performance on separate

axes

The Global Community for Academic Advising

For example, if you wanted to measure a student’s level of knowledge regarding a

specific learning outcome, such as knowing departmental requirements, your vertical axis

may contain the criterion points for

*Knowledge

*Understanding

*Behavior

Assessment of Academic Advising InstituteAssessment of Academic Advising Institute

While your horizontal axis may contain the degrees of performance.

Points can include:•Adequate/not adequate•Excellent/competent/developing•Range of numbers

The Global Community for Academic Advising

Such that the beginning of your rubric would look like:

Excellent Competent Improving Developing

Knowledge Criterion

Understanding Criterion

Behavior Criterion

The Global Community for Academic Advising

You would then add descriptors for the intersection of each criterion point and each degree of performance

For example, a descriptor for the criterion Knowledge may be “Ability to communicate the required courses for the major”

For the criterion Understanding, it may be“Ability to comprehend the reasoning behind the requirements”

And for the criterion Behavior it may be“Ability to schedule/enroll in appropriate major courses”

Assessment of Academic Advising InstituteAssessment of Academic Advising Institute

Adding these descriptors for your criterion points, your rubric would evolve into

Excellent Competent Improving Developing

KnowledgeAbility to communicate the required courses for the major

UnderstandingAbility to comprehend the reasoning behind the requirements

BehaviorAbility to correctly schedule/enroll in appropriate major courses

The Global Community for Academic Advising

Descriptors would also be added for the degrees of performance for each criterion as well

for example, the descriptor:For Excellent for the criterion Knowledge may be“Can delineate all requirements for the major”

For Competent regarding the criterion Knowledge, it may be“Can delineate most of the requirements for the major”

For Improving it may be“Can delineate some of the requirements for the major”

For Developing for the criterion Knowledge it may be“Cannot delineate any requirements for the major”

Assessment of Academic Advising InstituteAssessment of Academic Advising Institute

Adding these descriptors to the degrees of performance for the criterion Knowledge would result in the rubric looking like

Excellent Competent Improving Developing

KnowledgeAbility to communicate the required courses for the major

Can delineate all requirements of the major

Can delineate most requirements for the major

Can delineate some requirements for the major

Cannot delineate any requirements for the major

UnderstandingAbility to comprehend the reasoning behind the requirements

BehaviorAbility to correctly schedule/enroll in appropriate major courses

The Global Community for Academic Advising

You then continue to add descriptors for each degree of performance relative to each of the remaining criterion points of Understanding and Behavior

Your final rubric may look like:

Assessment of Academic Advising InstituteAssessment of Academic Advising Institute

Excellent Competent Improving Developing

KnowledgeAbility to communicate the required courses for the major

Can delineate all requirements of the major

Can delineate most requirements for the major

Can delineate some requirements for the major

Cannot delineate any requirements for the major

UnderstandingAbility to comprehend the reasoning behind the requirements

Can clearly describe reasons/rationale for required courses

Can clearly describe many reasons for the required courses

Can clearly describe several reasons for the required courses

Cannot describe any reasons for the required courses

BehaviorAbility to correctly schedule/enroll in appropriate major courses

Enrolls in appropriate courses to fulfill major requirements

Enrolls in several appropriate courses to fulfill major requirements

Enrolls in a few appropriate courses to fulfill major requirements

Has no appropriate courses on schedule

The Global Community for Academic Advising

Rubrics may also include just one criterion point for a very specific task(e.g., attend a workshop)and just one dichotomous level of performance(e.g., the student did or didn’t attend)

Assessment of Academic Advising InstituteAssessment of Academic Advising Institute

Construction of a rubric requires:

- reflection on the overall objectives, goals, and outcomes of your program, and

- identification of what you hope will be accomplished via academic advising

The Global Community for Academic Advising

The Global Community for Academic Advising

Evaluating Rubric Data• One way to evaluate data from a rubric

– The % of students that fit into each degree of performance for each criterion.

• Another way to evaluate data from a rubric– develop a score range to see what the numerical average is

for each criterion point.• •Excellent = 4• •Competent = 3• •Needs More Info = 2• •Not Aware = 1

– This could be tracked longitudinally to establish a benchmark

The Global Community for Academic Advising

Rubric – Withdrawal (732)Completed by advisors after appointments

– What are the % of students in each degree of performance for the first criterion – explaining withdrawal?

– What is the mean (average) degree of performance on students communicate withdraw deadline?

Average is 3.09 on 4.0

Rubrics are a good way to allow a quick evaluation of a performance or

demonstration of knowledge, but should be utilized as just one of multiple measures

for any given performance or demonstration of knowledge

Assessment of Academic Advising InstituteAssessment of Academic Advising Institute

Resources

Maki, P. L. (2004). Assessing for Learning; Building a Sustainable Commitment Across the Institution. Sterling, VA: Stylus

Mueller, J. (2006). Authentic Assessment Toolbox. http://jonathan.mueller.faculty.noctrl.edu/toolbox/rubrics.htm. Retrieved 1/26/06.

Stevens, D. D., and Levi, A. J. (2005). Introduction to Rubrics. Sterling, VA: Stylus.

The Global Community for Academic Advising