Introduction to Integrating Cultural Competence Into Student Learning Outcomes and Assessment Webinar Presented by Educational Testing Service Institute.

Introduction to Integrating Cultural Competence Into

Student Learning Outcomes and Assessment

Webinar Presented by

Educational Testing ServiceInstitute for Evidence-Based Change

Ross Markle, Ph.D. Senior Research and Assessment Advisor,

Global Education Division, Educational Testing Service (ETS)

Javarro Russell, Ph.D. Senior Research and Assessment Advisor,

Global Education Division, Educational Testing Service (ETS)

Our Panelists Today

Brad C. Phillips, Ph.D. President/ CEO

Institute for Evidence-Based Change

What Will This Webinar Accomplish?A Taste of…

• Learning practical approaches to develop effective student learning outcomes (SLOs)• Discovering ways to integrate cultural competencies• Gaining insights into how student learning outcomes can be easily

measured• Learning about SLO assessment from experts

The Problem

Why Now?

• Changing face of California• Epic opportunity to make a difference• Pressure to improve outcomes

“If you treat an individual … as if he were what he ought to be and could be, he will become what he ought to be and could be.”

Johann Wolfgang von Goethe

Culture Defined

For the purposes of the Intercultural Studies Project, culture is defined as the shared patterns of behaviors and interactions, cognitive constructs, and affective understanding that are learned through a process of socialization. These shared patterns identify the members of a culture group while also distinguishing those of another group.*

*Center for Advanced Research on Language Acquisition: http://www.carla.umn.edu/culture/definitions.html

Cultural Competence Defined

Cultural competence is a set of congruent behaviors, attitudes, and policies that come together in a system, agency or among professionals and enable that system, agency or those professions to work effectively in cross-cultural situations.

*Cross L.

Cultural Competence Applied*

Essential elements contribute to a system's, institution's or agency's ability to become more culturally competent: which include:• Valuing diversity• Having the capacity for cultural self-assessment• Being conscious of the dynamics inherent when cultures interact• Developing institutionalized culture knowledge• Developing adaptations to service delivery reflecting an

understanding of cultural diversity *Cross L. (edited)

Cultural Competence Applied*

Essential elements contribute to a system's, institution's or agency's ability to become more culturally competent: which include:• Valuing diversity• Having the capacity for cultural self-assessment• Being conscious of the dynamics inherent when cultures interact• Developing institutionalized culture knowledge• Developing adaptations to service delivery reflecting an

understanding of cultural diversity *Cross L. (edited)

Student Equity Categories

• Race/ethnicity• Gender• Age• Veterans• Foster Youth• DSPS• Socioeconomic status

Cultural Competence Categories• Other Categories• Native language• Generation• Political ideology• Religious and spiritual beliefs• Family status• Marital status• Parental status• Sexual orientation• Body type• Learning ability/disability• Learning styles• Mental health• National citizenship• Others?

Student Learning Outcomes: Critical or…

Many see this work as a worthwhile activity, but…• Others see this work as put upon them or busy work

“check the box” and want to continue doing what they have always doneFurthermore,• SLOs are not well integrated in to adjunct/part-timers work• Often do not link up to what is done in the classroom

Bloom’s Taxonomy

New (and practical) Ways to Develop Student Learning Outcomes – Adelman’s Rules• We know that an SLO has:

• An operational verb• A concrete noun• An activity used to demonstrate the learning (formative or summative)

• It should not have• The words “Ability to” or “Able to”• Verbs that are really nouns (display, estimate, etc.)• Abstract verbs (appreciate, awareness, etc..)• Qualifiers (suitable, creative, reasonable, etc.)• A lack of a way to measure during the learning experience• Phrases where there is a lack of agreement such as “think critically”

How Can This Work Begin?

• Develop a Discipline Core• Every Discipline should work to develop statements about:

• Who we are• What we study• What we do• What happens to our graduates

• Examine assignments and assessments• What learning outcomes are embedded?

• From this core work, SLO’s emanate

Why Use Adelman

• More intuitive• Can start with what you already do – your existing assignments and

assessments• Categories are behavioral

• To Imagine a Verb: The Language and Syntax of Student Learning Outcome Statements, Adelman, C. February, 2015

Write Statements Under 20 Different CategoriesStart with your assignments and assessments – what are you asking student to do?• Acquisition and preparation of tools

• Access, acquire, collect, gather, locate, etc…

• Combining ideas, materials and observations…• Assimilate, connect, integrate, link, etc…

• Formatting materials, data, information and materials• Arrange, assemble, collate, organize, etc…

• Developing inquiry skills• Examine, hypothesize, investigate, etc…

• Etc.• http://learningoutcomesassessment.org/documents/Occasional_Paper_24.pdf

Lets Do a Very Basic Example..

• Current way to write

• New way to write integrating Adleman’s work

A Student Learning Example Blending Adelman and Cultural CompetenciesToo general Students will be able to apply appropriate algebraic methods.

No way to measure Students will be able to solve word problems using the correct algebraic equations

with one unknown variable.

Better Students will be able to solve word problems using the correct algebraic equations

with one unknown variable.

Example – The City Bus*• “It costs $1.50 for John to travel each way on the city bus. A transit

system "fast pass" costs $65 a month. Which is the more economical way to get to work, the daily fare or the fast pass?

*Thompson, A and Cuseo, J.B.

Solving the Problem

• Daily fare• Fast pass• Other

• Does income come into play here?• What is a “fast pass”?• What about urban, rural, suburban environments?

How can the question be rephrased?

Revised Example – The City Bus*• “It costs $1.50 for John to travel to work each way on the city bus.

John works one job, Monday through Friday. A transit system "fast pass" (monthly pass) costs $65 a month. Which is the more economical way to get to work each day, the daily fare or the fast pass?”

Cultural Competence: General Principles

• Holding high expectations for all students and communicating those to them – syllabus • Using on-going and diagnostic assessment• Being mindful about test items and how they can be interpreted• Building connections between faculty and students• Allowing students to build connections with one another (intentionally

selecting the group)• Using time flexibly for student needs – understanding what is going on

with that student

Cultural Competencies: Specific Principles

• Remove norms from assignments and test questions • Omit what is not relevant to the content• Add what is necessary to help contextualize the content• Vary classroom activities, materials and assessments to account for different

cultural and learning styles. For example, in some cultures it is not acceptable to speak up in a group. So if group discussions are an expectation, these students will be at a

disadvantage. • Create multiple materials – supplemental materials and different perspectives• Develop multiple ways of assessment, e.g. essay, discussion, test, etc.

A Culturally Competent Lens for Measuring Student Learning

Designing Measure of Student Learning at ETS

Step 1 in developing a snapshot of student learning with a culturally competent lens

Decide what you want to say about what your students learn.

Writing Skills

Critical Thinking

Academic Skills

Information Seeking Skills

Designing Measure of Student Learning at ETS

Step 2 in developing a snapshot of student learning with a culturally competent lens

Decide what evidence of student learning is needed.

Knowledge Based

AssessmentSurvey

Rubric Portfolio

Designing Measure of Student Learning at ETS

Step 3 in developing a snapshot of student learning with a culturally competent lens

Develop questions that allow you to observe student learning.

FOCUS ON FAIRNESS BY ATTENDING TO…

• cultural diversity of the United States• diversity of background, cultural traditions and viewpoints found in both U.S.

and international test-taking populations• changing roles and attitudes toward groups in U.S. society• contributions of various groups (including ethnic and minority groups,

individuals with disabilities and women) to the history and culture of the United States and the achievements within these groups

• role of language in setting and changing attitudes toward various groups

3 Tips for Improving Cultural Competence in Designing Measures of Student Learning1. Be able to identify student characteristics that may impact student

learning and performance on assessments

2. Consider multiple pathways for students to demonstrate knowledge gained as a result of your educational programming

3. Consider soliciting feedback from students on your assessments

The Assessment Process

31

Establish Student Learning Outcomes (SLOs)

Identify or Develop Measures

Gather Data

Analyze DataInterpret Results

Use Results: Improvement or Demonstration

An Example from LaGuardia Community College

3232

Established SLOs in Literacy, Info. Literacy, Oral Comm., Quant.

Literacy

Selected assessments: modified AAC&U Value

Rubrics to evaluate student artifacts

Gathered data through random

samples of e-Portfolios

Analyzed data: Compared students

with <25 and >45 credits

Interpreted results through reports and discussions at faculty

meetings

Used results to revise capstone course; Scaffold

curriculum

www.learningoutcomesassessment.org/documents/LaGuardiaCC.pdf

Where does Cultural Competency fit in?• SLOs and Measures

• Ultimately must ask: are all our students learning equally?

• Example: considering first-generation college students• Gather: Are we assessing enough first-generation students?• Analyze: How will we compare the performance of first-generation students?• Interpret: What approaches can we take to improve learning for first-

generation students?

33

Questions

Contact Information

Ross [email protected]

Javarro Russell [email protected]

Brad C. [email protected]