Selecting Appropriate Assessment Measures for Student Learning Outcomes October 27, 2015 Cathy Sanders Director of Assessment Office of Assessment and.

Selecting Appropriate Assessment Measures for

Student Learning Outcomes

October 27, 2015Cathy Sanders

Director of AssessmentOffice of Assessment and Accreditation

Workshop Learning Outcomes

Upon conclusion of the workshop, participants will understand the difference between:• formative and summative assessments; • direct and indirect measures; and• traditional and authentic assessments.

Participants will also understand how the expected level of learning affects the selection of appropriate assessment measures for student learning outcomes.

Formative Assessment

Formative assessment is assessment that is conducted by instructors during the learning process for the purpose of monitoring student learning and modifying teaching and learning activities to improve student learning.

Formative Assessment is Useful for:

• getting a clearer idea of where students are, and thus, where to begin instruction;• revealing how well the students are following the lesson in

progress; and• uncovering gaps in understanding before they become

serious impediments to further learning.

Summative Assessment

Summative assessment is assessment that occurs at the end of a program to evaluate student learning by comparing it against some standard or benchmark.

Summative Assessment is Useful for:

• guiding curriculum revision;• evaluating the effectiveness of educational programs; and• responding to external reporting requirements (e.g., professional and

regional accreditors).

The focus of this workshop is on summative, program-level assessment (e.g., BS in Economics).

OAA will be offering some formative, classroom assessment workshops in the spring.

Direct Measures

• Direct measures are assessments that require students to demonstrate the application of knowledge, skills and abilities.• Examples of direct measures are written papers,

presentations, recitals, performances and projects.• At least one direct measure is required for each program

student learning outcome.•Multiple measures are encouraged.

Indirect Measures

• Indirect measures do not require students to demonstrate the application of knowledge, skills, and abilities.• Indirect measures either assess opinions or thoughts (e.g.,

surveys and focus groups) or provide indicators of program effectiveness (e.g., retention and graduation rates).• Indirect measures are optional for program student learning

outcomes.

Traditional Assessments

• Traditional assessments are forced-choice assessments such as multiple-choice, true-false, fill-in-the-blank, and matching tests.• Students typically select an answer or recall information to

complete the assessment.• These tests can be standardized or teacher-created.

Authentic Assessments

• A form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills. • Authentic assessments ask students to analyze, synthesize and apply

what they have learned in a substantial manner, and to create new meaning for students in the process.

Authentic Assessment Toolbox Website (Jon Mueller)http://jfmueller.faculty.noctrl.edu/toolbox/whatisit.htm#definitions

Traditional Assessment Model

In the traditional assessment model, the curriculum drives assessment. • The body of knowledge is determined first. • That knowledge becomes the curriculum that is delivered.

• Then the assessments are developed and administered to

determine if acquisition of the curriculum occurred.

Authentic Assessment Model

In the Authentic Assessment Model, assessment drives the curriculum.• The instructor first determines the tasks that students will

perform to demonstrate their mastery of the student learning outcomes.• Then a curriculum is developed that will enable students

to perform those tasks well which includes the acquisition of essential knowledge and skills.

Defining Attributes of Traditional and Authentic Assessments

Traditional

• Selecting a Response• Contrived• Recall/Recognition• Teacher-structured• Indirect Evidence

Authentic• Performing a Task• Real-life• Construction/Application• Student-structured• Direct Evidence

Aligning the Assessment Measure with the Learning

• Expected level of learning also affects the design of an assessment measure.

• Since program-level student learning outcomes are assessed at the mastery level, more complex assessment measures are appropriate.

• Bloom’s Revised Taxonomy is useful in developing assessment measures that align with expected level of learning.

Background on Bloom’s Taxonomy

• Created in 1956 by Benjamin Bloom, an educational psychologist and several of his colleagues that were interested in improving student learning.

• Revised later by a former student of Bloom, Lorin Anderson to better fit educational practices in the 21st century.

• Consists of hierarchical frameworks where each level is subsumed by the higher, more complex level.

Bloom’s Revised Taxonomy

RememberingThe learner is able to recall, restate and remember learned information.

Verbs associated with Remembering:

• Describe• Find• Identify• List• Recall

• Retrieve• Name• Locate• Recognize• Reproduce

Can students recall information?

Northern Illinois University, Faculty Development and Instructional Design Center

UnderstandingStudent grasps meaning of information by interpreting and translating what has been learned.

Verbs associated with Understanding:

• Classify• Compare• Exemplify• Explain• Translate

• Infer• Interpret• Paraphrase• Summarize• Expand

Can students explain ideas or concepts?Northern Illinois University, Faculty Development and Instructional Design Center

ApplyingStudent makes use of information in a context different from the one in which it was learned.

Verbs associated with Applying:• Implement• Carry out• Illustrate• Solve

• Use• Execute• Examine• Classify

Can students use the information in another familiar situation?

Northern Illinois University, Faculty Development and Instructional Design Center

AnalyzingStudent breaks learned information into its parts to best understand that information.

Verbs associated with Analyzing:

• Attribute • Compare• Deconstruct• Find

• Integrate • Organize• Outline• Structure

Can students break information into parts to explore understandings and relationships?

Northern Illinois University, Faculty Development and Instructional Design Center

EvaluatingStudent makes decisions based on in-depth reflection, criticism and assessment.

Verbs associated with Evaluating:

• Check• Critique• Detect • Experiment• Prioritize

• Hypothesize• Judge•Monitor• Test• Recommend

Can students justify a decision or a course of action?Northern Illinois University, Faculty Development and Instructional Design Center

CreatingStudent creates new ideas and information using what previously has been learned.

Verbs associated with Creating:

• Construct• Design• Devise• Invent

• Compose • Plan• Produce• Propose

Can students generate new products, ideas, or ways of viewing things?

Northern Illinois University, Faculty Development and Instructional Design Center

Sample Questions Stems for Lower-level Learning: Understanding

Question stems for Bloom’s “Understanding” level of learning:• How would you classify…?• Compare and contrast…•Which statements support…?• How would you summarize…?•What facts or ideas show…?

(Pohl, 2000)

Sample Question Stems for Higher-level Learning: Creating

Question stems for Bloom’s “Creating” level of learning:• Design a…to…• Devise a possible solution to…• Develop a proposal which would…• Construct a new model that would change…• Formulate a theory for…

(Pohl, 2006)

Summary: for Program-Level Student Learning Outcomes…• Use direct measures that required students to demonstrate the

acquisition of knowledge, skills and abilities.

• Use authentic measures that ask students to analyze, synthesize and apply what they have learned and to create new meaning.

• Develop the curriculum around the student learning outcomes that department faculty have identified as most important.

For Program-Level Student Learning Outcomes…• Determine the level of learning expected in the student learning

outcome (e.g., application, analysis, evaluation or creation).

• Design assignments, assessments and analytic scoring rubrics with the expected level of learning in mind.

• Reference verbs and question stems for each of Bloom’s levels of learning provided in your workshop folder.

Questions and Comments