No Advanced User Left Behind - Discovery for All Beth Namei & Caroline Muglia University of Southern California Libraries ProQuest User Summit | Chicago, IL | October 8, 2015
No Advanced User Left Behind - Discovery for All
Beth Namei & Caroline Muglia University of Southern California Libraries
ProQuest User Summit | Chicago, IL | October 8, 2015
What we’ll cover today
● Timeline of USC’s implementation of Summon● Portrait(s) of Advanced Users● Results & Snapshots of 3 user studies and search query
analysis conducted from 2010-2015● Pain Points● Recommendations● Q&A
Why USC launched Summon
● Better discoverability
● Relevant results to our users
● Richer user experience with the library’s website
Novice researchers are happy with Summon!Freshman: “For my research papers, I always come to the USC website and Quick Search is really helpful…”Freshman:“The first thing I would do is probably Quick Search it… that’s my first step, that’s the easiest, it’s right there.”Freshman:“I use Google Scholar, but I find that it’s easier to use Quick Search because Google Scholar doesn’t link you to USC LIbraries all the time.”
Source: https://docs.google.com/document/d/1ajpryZbNmmFFceimG1yHgBnO5F6Icl5lr9eXb0d29SE/edit?ts=5612e99d
Why discovery vendors and academic librarians (should) care about advanced users?
Who are advanced users?They have…
● Specialized subject knowledge ● Familiarity with library's collections and resources● Experience researching in their discipline● Commitment to in-depth research● Sophisticated searching skills● Mastered and adapted to less than ideal search tools/systems
They want…● Precision search tools that allow them to hone in on sources● To search for known items using objective info. (author name, projects, article, book or journal titles, etc.)
Sources: “Not in Love, or Not in the Know? Graduate Student and Faculty Use (and Non-Use) of E-Books” - doi:10.1016/j.acalib.2012.08.005; http://ejournals.library.ualberta.ca.libproxy2.usc.edu/index.php/EBLIP/article/view/1802/3366 and http://www.emeraldinsight.com.libproxy1.usc.edu/doi/pdfplus/10.1108/07378830710840509
Advanced users are the majority
http://clipgid.com/professor-teaching-clipart.html http://www.supercollege.com/images/grad_shadows_white_2.jpg
http://img11.nnm.me/9/0/b/6/d/d096d47ae3b91b239668df781ea.jpg
Advanced users are embedded Quality Control
Image from: http://qainsight.net/category/qa-comics/
Advanced users influence novice users
Image from: http://blog.ocad.ca/wordpress/fcdc/files/2014/10/college-clip-art-9iRq8p4ie.gif
Advanced users are decision makers
Image from: http://agilevietnam.com/2012/12/28/understanding-decision-making/
Discovery-centric Library Homepages
2010-2015
2010 - Pre-Summon homepage
July 2010 - Summon is launched as “Quick Search” - becomes the default tab
July 2012 - Summon-centric homepage (catalog tab is removed)
From 2012-2015, the catalog was still a visible option, just less prominentvia a drop-down menu
an icon below the main Summon search box
at the same time we also launched SIRSI Enterprise - a next-generation catalog
July 2015: Complete website redesign & rebranding of our Summon instance
Catalog is less visible than ever before
Catalog vs. Summon Usage @ USC (2008-2015)
3 user studies (2013-2015)
Usability testing of homepageFall 2013
● Interviewed 13 users○ 4 undergraduates (3 freshman, 1 senior)○ 4 faculty○ 5 graduate students
● Accessed current homepage● Accessed mock-up of new homepage
Trust
Image from: http://www.gratis-malvorlagen.de/science-fiction/wissenschaftler-2/
“I don’t trust Quick Search. I only use it if it is absolutely necessary….In order for this site to be successful, you have to have success after success after success because people...aren’t going to waste their time…I will admit to you that the odds are slim and none that Quick Search will get me anywhere close to what I want….If I need to search something I don’t go to any USC search engine….I go to Google where I can get things ten times as fast.”
Advanced user #1: Religion Professor
Reliability
Advanced user #2: English Professor
“I could throw an author and title in [Summon] but I know that works very erratically….I often get inundated by irrelevant things…”
Image from: http://www.flickriver.com/photos/mitopencourseware/sets/72157621971051805/
Annual InSync Library Survey Fall 2013
2,736 survey responses
● 2,256 undergraduate and graduate students
● 235 faculty
“Journal articles are SO much easier to find using google if you already know the citation title.”
Need for Precision Tools
Advanced user #3: Cinematic Arts Graduate Student
“Your catalog seems to have problems returning accurate results. For example, searching the exact title of a book sometimes shows that you do not own the book, when you actually do have it.”
Advanced user #4: Law Professor
image from: http://famonthelam.com/wp-content/uploads/2015/02/dc8oKMgce.jpeg image from: http://blogs.bedfordstmartins.com/bits/files/2012/02/digi-liz.jpg
Problems finding known items“Please, please, please fix the search function for books….The results are awful. I'll type in a book's title, and the book itself will be on page two after pages of reviews. Or I'll be looking for a journal article and it'll be impossible to find in a swarm of totally unrelated results.” Advanced user #5:
Undergraduate, Junior
“Make it easier to search for a book by author through the website. Searching by title is easy. It's difficult to search by author.”
Advanced user #6: Social Work Graduate Student
Image from: http://jeannelking.com/wp-content/uploads/2014/08/Closeup-of-good-enough-suspicious-facial-expression.jpg
Image from http://www.huffingtonpost.com/wait-but-why/generation-y-unhappy_b_3930620.html
Search Log Analysis (2014)
http://bit.ly/Summon-relevance
2014 ACRL Study of Known Item Search Queries in Summon Objectives: To compare the relevancy of results when searching for known items in Summon to Google and Google Scholar.
To better understand user behavior and experience when searching for known items via our main search option.
Sample Size: 384 known item queries were examined → in order to compare the relevancy across 3 search tools we reduced these to 278 (by eliminating non-scholarly items--videos, sound recordings, etc.--and items USC did not own)
What kinds of known items are USC users searching for?
n= 384
How are they searching for these known items?
n= 384
Defining Relevant Results● Relevant = a match for the known
item shows up 1st or 2nd in the list of results.
● Partially Relevant = a match for the known item is found 3rd-10th in the list of results.
● Not relevant = known item is not listed in the first 10 results or no results were returned.
Image from: http://relevantinfo.org/wp-content/uploads/2013/09/good-o-meter.jpg
Relevancy of Results for Scholarly Known Item Searches - March 2015
28%
n= 278
Relevancy of Results for Scholarly Known Item Searches - June 2015
21%
n= 278
in other words...
between 177,500 - 288,450 known item searches did
not lead to relevant results in 2014!
Image from: http://cache1.asset-cache.net/xc/538990835.jpg?v=2&c=IWSAsset&k=2&d=69V2Nb-aNPGQep7gMBCUQ2J-GKIQ445o7LyHHB91Fdj6ANhQcooFq0hMe_YiRJd10
failed results = dead ends for users
Image from: https://gigaom.com/2011/03/28/opinion-preaching-to-the-green-choir-a-dead-end/
Causes of “failed” searches After April 2015 Summon Release
n=57
2 Pain Points (Dead Ends): Linking Problems & User Errors
n=57
360Link 2.0We will be launching the new 360Link 2.0 Sidebar to provide more intuitive and visible help when users are faced with error messages or broken links.
Not a perfect solution though...
360Link Sidebar Usability Study September/October 2015
Graduate student: When faced with an error message after clicking a Full Text Online link from Summon:
“When this happens I assume I did something wrong….I would do a Google search to make sure I have the correct article information”
Instead of reporting a problem when faced with these linking problems, she would request the item via interlibrary loan.
User errors● 17% of all queries had 1-2
(47/278) user input errors○ 36% (17/47) of these led
to failed searches ● Types of errors:
○ Typos (transpositions, substitutions, omissions, insertions)
○ incorrect use of boolean Image from: http://human-error.sarkisozlerik.com/human-error/a-lifetime-by-design.html
Help wanted!“Did you mean” suggestions only showed up 8 times (3% - 8/278) for all searches in our study. ● And only for 4 of the (47)
searches with user errors. ● And of these, only 2 of the
suggestions actually led to the known item.
Image from: http://www.debrafine.com/wp-content/uploads/2015/01/image004.gif
Recommendations for Librarians● Decide whether or not to include ALL
databases/vendors in your discovery system○ Especially those that do not support direct linking or
have persistent linking problems○ Be a squeaky wheel! Make sure vendors know that
addressing these problems is a priority.
● Upgrade to 360Link 2.0 and do usability testing to make sure wording is actually helpful to ALL users
Recommendations for ProQuest● Improve query
reformulation suggestions○ consider automatically
redirecting errors (like Google)?
● Continue to improve the relevancy of results for both known item and topic searches
Other Recommendations for ProQuest● Accommodate copy and pasted and formatted citation queries
○ 4% (17/384) of the queries copied and pasted 3 or more citation elements (not necessarily something advanced users do but other studies have found this happening more and more)
● Offer more browsing options● Improve discovery for non-textual known item searches (videos,
sound recordings, etc.) ○ 78% (7/9) searches for videos USC owns did not find relevant
results (related to average # of words in film titles and minimal/poor metadata)
“digital abundance [is] making it harder for [researchers] to feel confident that they had found everything they would need to find in order to reach and defend their research conclusions….Would it be possible...to develop a discovery tool that is designed not to find the best items but rather to provide some assurance that you hadn’t missed something?”
Keep imagining the future of discovery
Roger Schonfeld, June 1, 2015 The Scholarly Kitchen blog
Questions? Comments?
Image from: http://www.opencolleges.edu.au/informed/features/student-driven-learning/
Thank you!Caroline Muglia, Collection [email protected]
Beth Namei, Reference & Instruction Librarian [email protected]