Direct Assessment Methods - University of Arizona Assessment Methods ... design of the education ... • Mixed-method and direct measures • Quantify behaviors, thoughts, ...

Direct Assessment Methods Measuring UA Student Employee Experiences

Lucas SchalewskiRebecca ClaasenMegan Forecki

Student Employee Development & Appreciation Committee

Student Employment as a Promising High-impact Practice

• 80% of students work while enrolled in college • Higher-persistence rates, increased sense of belonging,

and peer relationships• Considered a valuable practice employers look for • Most students understand work as experiential

How do we assess student employee learning and development?

Citi & Seventeen Magazine. (2013). College student pulse.King, J. E. (2002). Crucial choices: How students’ financial decisions affect their academic success. American Council on EducationKuh, G.D. (2008). High-impact educational practices. AAC&U.Cheng, D. X. (2003). How families of low-and middle-income undergraduates pay for college: Full-time dependent students in 1999-00.

Identify Goals & Objectives

Outcomes

Design Assessment

Method

Collect & Analyze Data

Communicate Findings

Refine & Change

SLO: Building managers will be able to explain all policies and procedures following training.

Direct Methods

Process employed to gather data which requires students to display student knowledge, behavior, or thought processes.

Direct evidence of student learning

Provides evidence of student learning

Indirect Methods

Process employed to gather data which asks students to reflect upon their knowledge, behaviors, or thought processes

Infers student learning

Provides evidence of students’ attitudes, perceptions, and experiences

Indirect Methods

Examples• Student satisfaction• Job placement rates• Focus groups• Retention rates• Exit interviews• Self-reported data

Advantages• Easy to administer• Clues to what should be

assessed directly• Great for values and beliefs

Challenges• Provide only impressions and

opinions, not hard evidence• May tell you what they think

you want to hear

Direct Methods

Examples• Pre and post-tests• Oral Presentation• Quizzes• Check-lists• Rubrics• Capstone project• Reflections

Advantages• Requires students to

demonstrate knowledge• Provides data that directly

measures outcomes• Supports formative assessment

Challenges• Time consuming• Students may not prefer this

method

SLO: Building managers will be able to explain all policies and procedures following training.

Indirect: Please rate your level of agreement with the following…I can explain the policies and procedures building managers implement when closing down the building.

Strongly agreeAgreeNeutralDisagreeStrongly Disagree

Direct:Summarize the polices and procedures building managers implement when closing down the building.

[Open text box]

Assessment of Alternative Breaks

Pre-post TestPlease state your level of agreement for the following:Strongly agree, Agree, Neutral, Disagree, Strongly Disagree

I prioritize the needs of thecommunity over my own.

I shop at places based on a company’s ethics.

What are the top 3 reasons you will participate in service in the future?

To understand social issuesPre-Test: 3

Post-test: 14

Because it makes me feel goodPre-test: 4Post-test:0

Communicate Findings

DenverCompare the healthcare

system in the US to a less

developed nation.

San Francisco What are the root causes of

hunger?:

TexasWhat are the benefits of

wildlife conservation,

and how do the centers benefit

their communities?

Las VegasHow can we improve the design of the

education system?

Open-ended questions:

Successes/Challenges

Office of Scholarships and Financial Aid

Content AssessmentHands-On Practice

Shadowing RubricSelf-Reflection

Supervisor PlacementExit Interview

Direct Assessment

0

10

20

30

40

50

60

70

80

90

100

Week 1 Week 2 Week 3 Week 4 Week 5

D2L Content Assessment

Attempt 1 Attempt 2

Content Assessment

Weekly Lab Assignment

Hands-On Practice

Direct Assessment

Application

Shadowing Rubric

Indirect Assessment

Supervisor Placement

Professional Goals Survey

Indirect Assessment

Weekly Reflection Assignment

Self-Reflection

Indirect Assessment

Exit Interview

Team Weigh-In

Indirect Assessment

Student Employee - Customer Service

SLO: Student desk representatives will be able to demonstrate accomplished customer service skills.

Advantages• Mixed-method and direct measures• Quantify behaviors, thoughts, and performances• Help students understand what is expected of them • Provide descriptive feedback to students for improvement• Promotes more accurate, unbiased, and consistent scoring

Rubrics

Student Employee Outcomes:Providing customer serviceSolving problemsActing professionally

Communicating Managing tasksInteracting with others

Summary & Recommendations

Summary• Student employment is a high-impact practice. • Direct and indirect assessments have strengths and

weaknesses, we need both• Rubrics are an accessible and useful tool to use for

direct assessment of student employees

Recommendations:• Have well-written outcomes for student employees• Use the best method(s) to assess outcomes• Involve students in the process, use the findings

• Seek out consultation from A&R if needed