Christie Heitkamp Manager, User Experience & Information Architecture Team, WorldCat Local & WorldCat.org Karen Calhoun Vice President, WorldCat and Metadata Services Online Catalogs: Designing with Users in Mind Webinar August 13, 2009
Jan 25, 2015
Christie HeitkampManager, User Experience & Information Architecture Team, WorldCat Local & WorldCat.org
Karen CalhounVice President, WorldCat and Metadata Services
Online Catalogs: Designing with Users in Mind
Online Catalogs: Designing with Users in MindWebinar
August 13, 2009
Polling –What Do You Think?Polling –What Do You Think?
1. Our staff understand what
our end users want from our
catalog
• A – True
• B – False
• C – Don’t know
By pirate johnnyhttp://www.flickr.com/photos/piratejohnny/2798872422/
More PollingMore Polling
2. My library’s end users are satisfied
with our online catalog
• A – True
• B – False
• C – Don’t know
By: sea turtlehttp://www.flickr.com/photos/sea-turtle/3181321172/
More PollingMore Polling
3. My library’s cataloging
practices and priorities reflect what end users want from our
catalog
• A – True
• B – False
• C – Don’t know
AN INTRODUCTIONAN INTRODUCTIONUse, User, and Usability Studies
AgendaAgenda
• Introduction to user experience
• Usability studies conducted on WorldCat.org and WorldCat Local over the past few years
• Meet the “users”
CreditsCredits
• Arnold Arcolio
• Mike Prasse
• OCLC Marketing group – Joanne Cantrell
• WorldCat Local Pilot partners
• User Experience (from wikipedia)
User experience design, most often abbreviated UX, but sometimes UE, is a term used to describe the overarching experience a person has as a result of their interactions with a particular product or service, its delivery, and related artifacts, according to their design
Usability & User ExperienceUsability & User Experience
User research methods we useUser research methods we use
• Usability Testing: 1:1 with a user completing a set
of tasks
• User Surveys: links or popups from our site
• Focus Groups: discussion groups with customers
or users
• Contextual Interviews: 1:1 observations and
interviews of users in their own environment
• Card Sorts: 1:1 activity with a user to understand
their mental model and information organization
• Soon to come at OCLC…usability testing with eye
tracking!
Wikipedia:Personas are fictitious characters created to represent the different user types within a targeted demographic that might use a site or product. Personas are useful in considering the goals, desires, and limitations of the users in order to help to guide decisions about a product, such as features, interactions, and visual design
A persona will help teams focus on designing an online catalog for a specific kind of user, instead of designing a system for any user (which usually means designing for a non-existent user)
Developing a persona for the online catalog searcherDeveloping a persona for the online catalog searcher
Who is using the catalog?Who is using the catalog?
Healthcare professional
Retired
Other (please specify)
Business professional
Teacher or professor
Student *
0% 5% 10% 15% 20% 25% 30% 35% 40%
7%
9%
12%
18%
19%
35%
Respondent TypeEnd Users Only (n=14,895)
Feb. 11 to June 27, 2009
What kind of students are these?What kind of students are these?
Graduate College Undergraduate College
High School Student
Middle School Student
Elementary School Student
Other 0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50% 48%
34%
11%
4%2% 2%
Type of Student (n=5,250)Feb. 11 to June 27, 2009
What kind of search? 80% known item searchingWhat kind of search? 80% known item searching
80%
20%
Looking for a Specific Item or Just Browsing When Searching World.org?
End Users Only (n=14,895)Feb. 11 to June 27, 2009
I searched for a specific item I was just browsing for information
What kind of item? 74% book searchesWhat kind of item? 74% book searches
74%
14%
1%2%
4%5%
Type of Specific Item Searched ForEnd Users (n=11,890)
Feb. 11 to June 27, 2009
Book Article CD DVD
Meet the “user” – primary personas of worldcat.orgMeet the “user” – primary personas of worldcat.org
Questions?Questions?
STUDY RESULTSSTUDY RESULTSOnline Catalogs: What Users and Librarians Want
With thanks to
Janet Hawk,
Joanne Cantrell,
Peggy Gallagher,
OCLC Market Research
Photo by allw3ndy
http://flickr.com/photos/allw3ndy/2757149584/
Credits: OnlineCatalogs Study
Online Catalogs: What Users and Librarians WantOnline Catalogs: What Users and Librarians Want
•End-Users expect online catalogs:•to look like popular Web sites
•to have summaries, abstracts, tables of contents
•to help find needed information
•Librarians expect online catalogs: •to serve end users’ information needs
•to help staff carry out work responsibilities
•to have accurate, structured data
•to exhibit classical principles of organizationhttp://www.oclc.org/us/en/reports/onlinecatalogs/default.htm
Objectives of our metadata quality researchObjectives of our metadata quality research
•Start over without assumptions about what “quality” is
•Identify and compare metadata expectations• End users
• Librarians
•Compare expectations of types of librarians
•Define a new WorldCat quality program …
•Taking into account the perspectives of all constituencies of WorldCat
• End users (and subgroups of end users)
• Librarians (and subgroups of librarians)
Research methodologies and demographicsResearch methodologies and demographics
• Focus groups
• Conducted by Blue Bear, LLC
• Three sessions: College students, general public, scholars
• Pop-up survey on WorldCat.org
• Conducted by ForeSee Results
• 11,000+ responses: Students (28%), educators (22%), business professionals (19%), other; mix of ages; 44% from outside U.S.
• Librarian survey
• Conducted by Marketing Backup
• 1,397 responses; North America (64%) and outside North America (36%); academic, public, special libraries; staff with roles in technical and public services, ILL, directors
What did we learn?
End-user focus group resultsWhat did we learn?
End-user focus group results
• Key observations:
• Delivery is as important, if not more important, than discovery.
• Seamless, easy flow from discovery through delivery is critical.
• Summaries and tables of contents are key elements of a description
• Improved search relevance is necessary.
What did we learn?
End-user focus group resultsWhat did we learn?
End-user focus group results
Casual Searchers
Undergraduates
Scholars
User-created lists not useful
User-created lists useful
User-created lists not useful
‘Shopper’ mentality: don’t even show me
what I can’t get now
User-contributed reviews and
ratings useful
Reviews and ratings useful if
authoritative
Strongest preference for
cover art
What did we learn?
End-user focus group resultsWhat did we learn?
End-user focus group results
•Other key findings of our study:
•Keyword searching is king, but …
• Advanced search (fielded searching) is useful
• Faceted browse is useful
• These help end users refine searches, navigate, browse, and manage large result sets
“End users enter a few short search statements into online IR systems. Generally, their queries bear two to four words.”—Karen Markey•Twenty-five years of end user searching, Part 1: Research findings. 2007. http://deepblue.lib.umich.edu/handle/2027.42/56093
•
What did we learn?
Pop-up survey suggestionsWhat did we learn?
Pop-up survey suggestions
Changes to help identify an item?
End users (n=7535)
End-user recommendationsEnd-user recommendations
• Improve search relevance
• Add more links to online full text (and make linking easy)
• Add more summaries/abstracts: Make summaries more prominent
• Add more details in the search results (e.g., cover art and summaries)
Librarian/staff survey resultsLibrarian/staff survey results
Recommended enhancements to WorldCatTotal librarian responses
Recommended enhancements to WorldCatTotal end-user responses
End-User Results: Recommended Enhancements
4
Librarian/Staff Results:
Highlighted Differences
14
1
What did we learn?
Librarians’ Perceptions of What End-Users Want
What did we learn?
Librarians’ Perceptions of What End-Users Want
Recommended enhancements to WorldCat
What did we learn?
Librarians’ Perceptions of What End-Users Want (2)
What did we learn?
Librarians’ Perceptions of What End-Users Want (2)
Recommended enhancements to WorldCat
Recommendations from librarian surveyRecommendations from librarian survey
• Merge duplicates
• Make it easier to make corrections to records (fix typos; do upgrades); “social cataloging” experiment—Wikipedia
• More emphasis on accuracy/currency of library holdings
• Enrichment—TOCs, summaries, cover art—work with content suppliers, use APIs, etc.
• More communication about what users say they want
Questions?Questions?
RESULTSRESULTSWorldCat Local Usability Testing
Usability Testing WorldCat LocalUsability Testing WorldCat Local
Test Sites from 4/2007 – present:• University of Washington• University of California
• Berkeley• Davis• Irvine
• The Ohio State University• Peninsula Library System, San Mateo, California• Free Library of Urbana• Des Plaines Public Library• Northeastern Illinois University• University of Illinois, Springfield
Usability Testing WorldCat Local – Phase 2Usability Testing WorldCat Local – Phase 2
Issues we wanted to test:
In general:• combining local, group, and global holdings;
well received by academics, needs branch information for publics
• including journal article and other licensed content;
welcomed single search for multiple kinds of content, but assumed it was comprehensive of library collection => metasearch
• simple search box; users expressed appreciation for a search box where they can “just type anything.”
• advanced search options; Used by about half of participants. Needs improvement to help user start a new search or refine an existing search.
Usability Testing WorldCat LocalUsability Testing WorldCat Local
Search Result Pages:• result ranking
Relevancy often accepted without question by most academic users. Trust library knows best how to do this for them. Sometimes surprised when an item chosen as relevant but the matching terms are not visible on search results.
• editions and FRBR, Most users do not search for a specific edition. Often the default edition chosen by WorldCat Local does not match expectations of what should be the default edition however (widest held vs. most recent).
• faceted browsing, Often noticed and praised, but not always used. Usage data indicates that they are highly used in worldcat.org. Format and Author are the two highest used facets.
• level of detail; Desire to see summary and TOC on search results. Desire to see why/where terms matched. Desire for indication of full text access, more location & availability information.
Usability Testing WorldCat LocalUsability Testing WorldCat Local
Item Details Pages:• getting print copies from local, group, and
worldwide collections; Local and group fulfillment was successful for academics. Publics were mostly concerned about what they could get at their local branch only. ILL was not of interest to publics or undergraduates, however usage indicates that ILL is highly used in academics.
• access to electronic resources; Users didn’t have a problem finding the links to electronic access, but they expect any link displayed will give them full text access. The display of WorldCat links along with local library links is problematic and confusing. When displayed with the open URL resolver, users often don’t know which is the best course of action to get to full text but are more likely to click on a link than a button to get to full text.
Usability Testing WorldCat LocalUsability Testing WorldCat Local
Item Details Pages:• collaborative and personal workflow features;
Anonymous ratings, reviews, reader recommendations, tags, and other user lists: For academics, need to know WHO is contributing this data and their reputation and identity. Reactions to anonymous user contributed data is largely negative. For publics, authority is less important. Users are more likely to contribute information.See Arnold Arcolio’s summary for more detailed information.
• navigation; Most use the browser “back” button to navigate back to results.
Usability Testing WorldCat LocalUsability Testing WorldCat Local
Item Details Pages:• Rearrangement of high-value bibliographic dataConducted surveys and card sorts to determine what information is most important when looking at an item in the catalog:
1. Title2. Author3. Summary/Abstract4. Publication Year5. Access to online content6. Publisher7. TOC8. Ability to see what is immediately avaialable9. Languages10. Editions
Detailed Record RedesignDetailed Record Redesign
Item Details Pages:
• Putting important Bib informationat the top of the page & Less important Bib Information lower on the page.
• Integration of library services on the page.
• Single place for display of electronicaccess.
• Fulfillment displayed with local/group• global availability.
Other improvements:• Grouped functionality based on user
expectations (card sorting): i.e. createda ‘toolbar’ for common functions likecite, print, email, bookmark.
• Introduced recommendations based onWorldCat data.
For more information about the usability studies we’ve done and a summary of results written by Arnold Arcolio, visit:
http://www.oclc.org/worldcatlocal/usability
Questions?Questions?
WHAT DOES IT MEAN FOR MY LIBRARY?WHAT DOES IT MEAN FOR MY LIBRARY?
Conclusion
What Does It Mean For Aligning What My Library Does with What Users Want?
What Does It Mean For Aligning What My Library Does with What Users Want?
By: David Wulffhttp://www.flickr.com/photos/dwulff/5357629/
Question 3 RevisitedQuestion 3 Revisited
•3b. My library’s cataloging practices and priorities reflect what end users want from our
catalog
• A – True
• B – False
• C – Don’t know
Two Starting PointsTwo Starting Points
1. Paying attention to what’s important about records
2. Aligning librarian and staff priorities with end user priorities• E-resources, books, media, unique digital
collections, special collection
• Redesigning work practices
Based on our two studies, what’s important about records?Based on our two studies, what’s important about records?
• Delivery information – item availability and links to content
• Basic bibliographic information (title, author, publication year, publisher, format, edition,
language)
• Evaluative content like summaries
• Fielded indexing (to support advanced search)
• Faceted browse (based on controlled forms names, topics, tags, etc.)
• Social features (for some audiences)
Follow upFollow up
• What did you think of the format of today’s session? (Webex with polling and chat)
• What did you think of the content of the presentation?
• What did you get out of the session that you can use in your work?
• What do you wish we had talked about?
Thank You!
Christie Heitkamp, heitkamc @oclc.orgKaren Calhoun, [email protected]
Thank You!
Christie Heitkamp, heitkamc @oclc.orgKaren Calhoun, [email protected]