Creating a User-Centered Culture of Assessment Stella Bentley and Bill Myers University of Kansas EDUCAUSE Southwest Regional Conference 2005.

Creating a User-Centered Culture of Assessment

Stella Bentley and Bill MyersUniversity of Kansas

EDUCAUSE Southwest Regional Conference 2005

Dilbert Cartoon

• Problem: Bicycle seats are hard. They hurt.

• Analysis: There must be something wrong with your pants

• Solution: Dorky pants.

Scott Adams, 7/18/94

North Central AssociationCriterion 2: Preparing for the

future

• Appropriate data and feedback loops are available and used throughout the organization to support continuous improvement.

• The organization provides adequate support for its evaluation and assessment processes.

Why Do Assessment

• Benchmark current services and use data to improve efficiency and effectiveness.

• Measure users’ perceptions and satisfactions with services.

• Establish what services and resources are needed and desired.

• Increase dialogue with users.

The User-Centered Organization

• Quality services and user satisfaction are goals recognized and shared by all staff.

• User services are given emphasis through ongoing assessment, implementation of user feedback, and empowerment of staff to respond to and solve problems.

• There is a high level of interaction with users.

We may have a GAP in our Assessment Activities

• What we think the customer wants• GAP• What the customer really needs

– Amos Lakos, ”Implementing a Culture of Assessment”, Conference presentation, 2004

Listen To Your Users!

Culture of Assessment

• Integrate assessment into the work process

• Build organizational framework to collect data, analyze facts, conduct research

• Focus on outcomes and impacts• Take action on what you measure

Culture of Assessment

• Continue what you do well• Concentrate on what you need

to improve• Experimentation is OK! – Take

risks!!!

Effective Assessment Methods

• Benchmarking: identification of best practices.

• Quantitative methods – statistics, questionnaires, user surveys.

• Qualitative methods – focus groups, interviews, unobtrusive observations.

Last year

1,154,460 people

visited our website

1,444,300 people

came through our doors

Did they find what they were looking for?

Did they get the help they needed?

We want to know

Our objective...

Creating a culture of assessment based on

evidence…

which leads to evidence in action.

Assessment council duties:

• Foster and promote culture of assessment

• Provide counsel• Advise libraries administration• Facilitate staff development• Act as clearinghouse

Assessment coordinator duties:

• Act as liaison to administration• Serve as initial point of contact• Convene and direct assessment

council• Maintain currency on assessment

practices and issues

• Work with colleagues to develop, conduct, and follow through on assessment priorities to demonstrate evidence in action

Some guiding questions...

What do we need to know?

Who can tell us?

How can we get the information?

What will it enable us to do?

What we have done:

• Developed and conducted staff development workshops on surveys & focus groups

• Reported on assessment activities• Brought in assessment experts from

other institutions• Participated in LibQUAL+• Participating in ECAR Study of

Students and Information Technology

• Worked with leadership to determine assessment priorities based on primary initiatives

• Sought to work collaboratively with other Information Services units and other campus partners

• Inventory data-gathering and assessment activities already in process

The specific charges of the Public Interface Design

Group (PID) are:

Review the public interface design of

enterprise Library systems and work with developers to continuously improve

service to users.

Review should include formal assessment

activities that assure our services meet our users needs and expectations.

Usability Study #1 of the Libraries Web Site

Goal:

Simplify the page

• Gathered and reviewed suggestions from the group

• Created mockups of new home page design

• Decided on a page design to test for usability

Web Usability Questions

Usability Study #2 of the Libraries Web Site

Average Question Completion Times

0

10

20

30

40

50

60

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Average

Question

Tim

e (s

ec)

Old

New

Freshmen Survey Results

Report Card

Libraries Home Page

Evidence in action

Questions & comments