User experience at Imperial: a case study of qualitative approaches to Primo usability studies

User experience at Imperiala case study of qualitative approaches to Primo usability studiesAndrew PreaterHead of Library Information Systems

@preater@imperiallibrary EPUG-UKI 2015

These slides:

x.preater.com/primoux

Slides and notes

Imperial College London

User Experience as a strategic priority

• Two iterations of user testing

• Redesign Primo UI

• Re-launch

• Groundwork for third iteration of

testing

Summary January – September 2015

Round one: usability. January 2015

• “Discount usability testing” (Nielsen, 2009)

• Student Placement: Sherif Khedry of UCL Qatar

Round two: user experience. April-May 2015

• Primo user experience: focused, qualitative approach

• Student Placement: George Bray of UCL DIS

@NexGenGB

Usability and user experience testing

Nielsen, J. (2009) ‘Discount usability: 20 years’. Available at: http://www.nngroup.com/articles/discount-usability-20-years/

Usability round 1: results

Library Search meets user expectations

Usability round 1: results

• Library Search meets expectations

• Discovery happens elsewhere

Discovery happens elsewhere

• Google et al.• Library databases• Citations• Lecturers• Friends

Usability round 1: results

• Library Search meets expectations

• Discovery happens elsewhere• User interface issues

UI issues

Where is the book?

UI issues

1.Extra click2.No emphasis

UI issues

?!

UI issues, ‘…and more’

User Experience testing: round 2

• 14 April planning

• 27 April UCL placement

student

• 5-8 May interviews & analysis

Research Questions: themes

• Purpose, construction, and use of search and resources

• Presentation of information: what matters to the user when selecting the right results?

• Grounded theory approach

• Coding of qualitative audiovisual data from:

1. Interviews2. Card sorting

Grounded theory as a UX method

• Acknowledge & work with our subjectivity as researchers

• Create abstract understanding from observing users

• Support a critical approach to library systems praxis

Constructivist grounded theory

Charmaz, K. (2014) Constructing grounded theory. 2nd edn. London: Sage

[edited]

[This slide originally showed a screenshot showing how we record video of the user and their screen]

Interview notes as research data

Facet card sorting

Questions for open coding“What is this data a study of?What do the data suggest?

Pronounce? Leave unsaid?From whose point of view?What theoretical category does this

specific [data] indicate?”

Questions to inform initial / open coding quoted from Charmaz (2014) p.116.Charmaz, K. (2014) Constructing grounded theory. 2nd edn. London: Sage

“What can it look like?”authority provides filtering techniqueage of journal is part of authoritycontext sensitivity of search: importance of age of material “depends on what you are doing

with it”human skill in judgement: “it is about your own judgement and experience”importance of recommendations from peers and seniorsaccumulation of small things peers say is importantlibrary is not a starting point for general infoproblem with our methodology: interviewee does not seem to want use Primo for thisgoogle is a starting pointguessing textbook names, there is often a textbook called the name of the discipline.use of location facetcampus as a factor “I want to stay in the Central Library”use of material facettitle: looks at titles first to judgeskimming through titles to see what is thereassessing if something is beginner-leveluncertainly: knowing i am not an expert

Questions for focused coding

Which work better overall as categories?Which give a better direction in developing

an overall theory from the data?How might you create a theoretical

framework about discovery user experience to help inform changes to the system? Which codes fit the data “snugly” & help you to do this?

Focused coding approach based on chapter 6 of Charmaz (2014) pp.138-161.Charmaz, K. (2014) Constructing grounded theory. 2nd edn. London: Sage

Summary findings

Results: main themes

• Searching should be as fast as possible

Results: main themes

• Searching should be as fast as possible

• Searching should be painless

Results: main themes

• Searching should be as fast as possible

• Searching should be painless

• Information is prioritised in selecting results

Redesign timeline

• July Complete redesign work on

sandbox

• 3 August acceptance testing

• 6 August cut over

• 6 August onward “many eyes” help

us find & fix bugs and issues

Presentation on browse results

Moving and ordering facets

Tweaking language, removing jargon

Tweaking language, removing jargon

• E-Shelf “Favourites”

• Advanced Search “More search

options”

• “Expand my results”

• Imperial house style changes

E-journal scoped search

1

Cleaner, tidier login menu

Cleaner, tidier login menu

• Changes in scope and

schedule

Key issues after cutover

• Changes in scope and schedule

• Advanced search

• A-to-Z dilemma

Key issues after cutover

A-to-Z dilemma

Browse e-journals by title

A-to-Z usage 2014-15

• Changes in scope and schedule

• Advanced Search

• A-to-Z dilemma

• Prioritising search results

• Inconsistent metadata

Key issues after cutover

Lessons learned

• Involve staff early in UX Project

• Staff development

• Acceptance testing adds value

• Going live != success

• Be prepare to justify design decisions

• Open communication

Summary lessons

[edited]

[This slide originally showed a screenshot of our Primo relaunch Trello board with current issues]

• Quantitative surveying

• “IOUX” UX investigation with

Bodleian Libraries. Focus on

PGT medics & NHS users

Round 2 ½ and Round 3 UX work

Charmaz, K. (2014) Constructing grounded theory. 2nd edn. London: Sage

Connaway, L.S., White, D., Lanclos, D., and Le Cornu, A. (2013) 'Visitors and residents: what

motivates engagement with the digital information environment?', Information Research, 18

(1), March. Available at: http://www.informationr.net/ir/18-1/paper556.html

Glaser, B.G. & Strass, A.L. (1967) The discovery of grounded theory. Chicago, IL: de Gruyter

Nielsen, J. (2009) ‘Discount usability: 20 years’. Available at:

http://www.nngroup.com/articles/discount-usability-20-years/

Pickard, A.J. (2004) Research methods in information. 2nd edn. London: Facet

Preater, A.J. (2015) ‘UX for the win! at #CityMash’, Ginformation Systems, 11 June. Available at:

http://x.preater.com/uxftw

Stohn, C. (2015) ‘How do users search and discover?’. Available at: http://

www.exlibrisgroup.com/category/UserStudiesWhitePaper

Select bibliography