Data-Driven Library Web Design: Making Usability Testing Work with Collaborative Partnerships Allison Cowgill, Head of Reference Amanda Dinscore, Public Services Librarian Patrick Newell, AUL for Information Technology and Electronic Resources Henry Madden Library California State University, Fresno All documents available at: http://www.slideshare.net/adinscore
Presentation from a workshop given at ACRL 2011 conference, Data-Driven Library Web Design: Making Usability Testing Work with Collaborative Partnerships
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Data-Driven Library Web Design: Making Usability Testing Work with
Collaborative Partnerships
Allison Cowgill, Head of ReferenceAmanda Dinscore, Public Services Librarian
Patrick Newell, AUL for Information Technology and Electronic Resources
Henry Madden Library California State University, Fresno
All documents available at: http://www.slideshare.net/adinscore
• The Library Study at Fresno State—Ethnographic study conducted by two anthropology professors• Study recommended that the Library’s web site should be should be redesigned
“Draw How You Feel When You Write a Paper.”
Drafting the Research Plan
Step 1: Create a Purpose Statement and Objectives
Purpose Statement: Should encapsulate the goals the team hopes to accomplish.
Example: “The purpose of this study is to determine if users can easily accomplish tasks required for research using the library’s web site.”
---------------------------------------------------------------------------Objectives: Use to develop the user tasks. Should
reflect actual user needs.
Example: “Determine the number of study participants who are able to search for and locate a book using the library’s web site.”
Activity: Create a draft
purpose statement and at
least 3 objectives.
Drafting the Research Plan
Step 2: Form the Team
Identify librarians and library staff who are sincerely interested in participating and support change.
Find collaborators outside of the library from academic departments such as Anthropology, Business, Computer Science, or Education. Everyone should be fully aware of the time and effort required.
Activity: Brainstorm
potential collaborators from
both within and outside your
library.
Drafting the Research Plan
Step 3: Identify User Tasks & Develop Questions
Consult with others (public service librarians, web team,…) • What is easy/difficult for users to do on our web site?• What do you spend time helping users do on our web site?
Create a list of tasks users are expected to perform and relate the tasks to the study objectives
Example tasks from our study: • Find a book title in our library• Find a book title through our patron-initiated borrowing system• Find a newspaper article• Find an article from a scholarly journal
Keep in mind the type of data you will collect & use
Task: An activity that fulfills an
information need
“All qualitative data can be coded quantitatively.”
Types of Data:Independent Variables: Variables you manipulate. Choose these based on your research questions.Dependent Variables (a.k.a. Outcome/Response Variables): Something you measure as the result of (based on the response to) the independent variables.
Quantitative Data: can be counted or expressed numerically Qualitative Data: nonnumeric information such as conversation, text, audio, or video.
Interval Likert scale data, SUS scores All descriptive statistics, t-tests, ANOVAs, correlation, regression analysis
Ratio Completion time, time (visual attention), average task success (aggregated)
All descriptive statistics, t-tests, ANOVAs, correlation, regression analysis
Drafting the Research Plan
Step 3: Identify User Tasks & Develop Questions
Examples:
Using the library web site, find one journal article on swine flu.
Show me where on the web site you can find help using the library.
The library has a page with resources organized by subject. Show me how to find the page with history resources.
Activity: Brainstorm at least 3 tasks based on
the objectives you created.
What data will you use to measure the
tasks?
Drafting the Research Plan
Step 4: Determine the Study Population
Who?
How many?
Drafting the Research Plan
Step 4: Determine the Study Population
The Size of Your Population or Sub-GroupWhy Sample?• To say something about a population• A statistically valid sample size allows you to generalize to a population from a sample
Confidence Level • Tells you how sure you can be• Represents how often the true percentage of the population who would pick an answer lies within the confidence interval
Confidence Interval • a.k.a. “margin of error”• A range that estimates the true population for a statistic
Technique Advantages Disadvantages
Random sampling •Theoretically most accurate.•Influenced only by chance.
Sometimes a list of the entire population is unavailable or practical considerations or prevent random sampling.
Systematic sampling •Similar to random sampling.•Often easier than random sampling.
The system can sometimes be biased.
Quota sampling •Can be used when random sampling is impossible.•Quick to do.
There may still be biases not controlled by the quota system.
Stratified sampling •Ensures large enough sample to subdivide on important variables.•Needed when population is too large to list.•Can be combined with other techniques.
Can be biased if strata are given false weights, unless weighting procedure is used for overall analysis.
Creating a script and instructions for moderators: • Helps them to clearly explain study procedures to subjects• Ensures uniformity throughout the process
Take advantage of collaborating teaching faculty’s expertise by enlisting their help to train student moderators.
Links: Moderator Instructions & Introduction Scripthttp://www.flickr.com/photos/lwr/4124641930/