LibQUAL+™

LibQUAL+™LibQUAL+™

GUGM Reference SectionGUGM Reference SectionMay 19, 2005May 19, 2005

Presentation byPresentation byBrian Mathews, Georgia Tech Brian Mathews, Georgia Tech

Caroline Killens, UGACaroline Killens, UGA

How to Listen to your Customers How to Listen to your Customers Using LibQUAL+™Using LibQUAL+™

How to Listen to your Customers How to Listen to your Customers Using LibQUAL+™Using LibQUAL+™

Definition of AssessmentDefinition of Assessment

In an academic environment assessment can be defined as In an academic environment assessment can be defined as “an ongoing process aimed at understanding and improving “an ongoing process aimed at understanding and improving student learning.student learning. It involves:It involves:

making our expectations explicit and public; making our expectations explicit and public;

setting appropriate criteria and high standards setting appropriate criteria and high standards for learning qualityfor learning quality;;

systematically gathering, analyzing, and interpreting evidence systematically gathering, analyzing, and interpreting evidence to determine how to determine how well performance matches those expectations and standards; andwell performance matches those expectations and standards; and

using the resulting information to document, explain, and improve performanceusing the resulting information to document, explain, and improve performance.. When it is embedded effectively within larger institutional systems, assessment can help When it is embedded effectively within larger institutional systems, assessment can help us focus our collective attention, examine our assumptions, and create a shared us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality of higher educationacademic culture dedicated to assuring and improving the quality of higher education (Thomas A. Angelo, (Thomas A. Angelo, AAHE BulletinAAHE Bulletin, November 1995, p.7). , November 1995, p.7).

The Importance of AssessmentThe Importance of Assessment

To evaluate your service.To evaluate your service.

To understand what patrons “really” want.To understand what patrons “really” want.

To guide the decision-making process.To guide the decision-making process.

To support administrative initiatives.To support administrative initiatives.

What is LibQUAL?What is LibQUAL? AA surveysurvey sponsored by ARL, as part of the New Measures initiative,sponsored by ARL, as part of the New Measures initiative, to to

augment the collection count and fiscal input measuresaugment the collection count and fiscal input measures of ARL of ARL that comprise the ARL Index and ARL Statistics. The survey is that comprise the ARL Index and ARL Statistics. The survey is comprised of “22 comprised of “22 questions + a box”questions + a box”

Is anIs an assessment toolassessment tool that letsthat lets customers judge qualitycustomers judge quality

Produces Produces outcome measuresoutcome measures on service quality and satisfaction of users.on service quality and satisfaction of users. Is a way of “listening” to usersIs a way of “listening” to users

Provides institutional data and reportsProvides institutional data and reports that enables you to assess that enables you to assess whether your library services are meeting user expectationswhether your library services are meeting user expectations

Provides libraries with comparable assessmentProvides libraries with comparable assessment information from information from peer institutionspeer institutions

Why Use LibQUAL+™?Why Use LibQUAL+™?

Great value with minimal effort.Great value with minimal effort.

To gain an objective understanding of your patron’s To gain an objective understanding of your patron’s unique needs and perceptions.unique needs and perceptions.

Provides comparative data with other libraries.Provides comparative data with other libraries.

Can be utilized with campus library reviews, SACS Can be utilized with campus library reviews, SACS reviews, accreditation reports, etc.reviews, accreditation reports, etc.

How To Prepare for LibQUAL+™How To Prepare for LibQUAL+™ Complete LibQUAL+™ questionnaire on-line to activate your account.Complete LibQUAL+™ questionnaire on-line to activate your account.

Determine a good source for your campus statistics!Determine a good source for your campus statistics!

Determine the best dates to conduct your survey.Determine the best dates to conduct your survey.

Determine if you need permission from your campus IR Board.Determine if you need permission from your campus IR Board.

Decide who to include in the survey and how it will be conducted.Decide who to include in the survey and how it will be conducted.Everyone on campus, random selection, etc.;Everyone on campus, random selection, etc.;If using a random sample, how will you generate your sample? Circulation files, If using a random sample, how will you generate your sample? Circulation files,

Human Human Resources files, Registrar’ files, etc.Resources files, Registrar’ files, etc.E-mail, website, mail paper copyE-mail, website, mail paper copy

Determine how much advertisement you want to conduct prior to the surveyDetermine how much advertisement you want to conduct prior to the surveyCampus newspapers, website, posters, alert departments, etc.Campus newspapers, website, posters, alert departments, etc.

Decide if you will use incentives.Decide if you will use incentives.iPods, no fines, money, sponsored gifts, etc.iPods, no fines, money, sponsored gifts, etc.

Prepare letter to be sent to participants. (sample provided by LibQUAL+™)Prepare letter to be sent to participants. (sample provided by LibQUAL+™)

What Happens While the Survey What Happens While the Survey Takes PlaceTakes Place

You can track the number of surveys completed for your institution You can track the number of surveys completed for your institution online in real time.online in real time.

You can begin reviewing comments immediately.You can begin reviewing comments immediately.

Send out reminders if it looks like things are slowing down.Send out reminders if it looks like things are slowing down.

Delete undeliverable addresses from your files so your reminders Delete undeliverable addresses from your files so your reminders will go only to active addresses.will go only to active addresses.

Have someone dedicated to answer questions by phone or e-mail – Have someone dedicated to answer questions by phone or e-mail –

RELAX! and let the survey work for you. RELAX! and let the survey work for you.

What Do You Get From What Do You Get From LibQUAL+™?LibQUAL+™?

LibQUAL+™ report within a month of your survey’s LibQUAL+™ report within a month of your survey’s closingclosing

SPSS and Excel files with all of the raw dataSPSS and Excel files with all of the raw data

Interactive files at the LibQUAL websiteInteractive files at the LibQUAL website

List of the comments made on your surveyList of the comments made on your survey

Access to reports from all other participating librariesAccess to reports from all other participating libraries

How The Survey WorksHow The Survey Works

Participants are asked to score each question in 3 sub-categories, based Participants are asked to score each question in 3 sub-categories, based on a scale of 1-9:on a scale of 1-9:

The The sub-categoriessub-categories for each question: for each question:

MinimumMinimum = “ = “what they needwhat they need” minimum acceptable level of service ” minimum acceptable level of service required byrequired by usersusers

PerceivedPerceived = “ = “what they havewhat they have” - the level of service users think/believe ” - the level of service users think/believe the library is providing (perceived) the library is providing (perceived)

DesiredDesired = “ = “what they wantwhat they want” - the level of service users want to have ” - the level of service users want to have (desired)(desired)

        When it comes to…..When it comes to….. My minimum service level My perceived service level My desired service My minimum service level My perceived service level My desired service low high low high low low high low high low high high

1)1)Employees who instill confidence users 1  2  3  4  5  6  7  8  9 1  2  3  4  5  6  7  8  9 1  2  3  4  5  6  7  8  9Employees who instill confidence users 1  2  3  4  5  6  7  8  9 1  2  3  4  5  6  7  8  9 1  2  3  4  5  6  7  8  9

Zone of ToleranceZone of Tolerance

GAPSGAPS

The scores from the 3 sub-categories can be utilized to show the The scores from the 3 sub-categories can be utilized to show the followingfollowing gaps: gaps:

1. 1. ADEQUACY GAPADEQUACY GAP : : how well the library is meeting the users’ how well the library is meeting the users’ “minimum” requirements/needs.“minimum” requirements/needs.

FormulaFormula: Perceived score minus the minimum score: Perceived score minus the minimum score

2. 2. SUPERIORITY GAPSUPERIORITY GAP : : how far the library is from meeting or how far the library is from meeting or exceeding the users’ desired level of serviceexceeding the users’ desired level of service

FormulaFormula: Perceived score minus the desired score: Perceived score minus the desired score

One way of viewing GAPSOne way of viewing GAPS(shows minimum perceived & desired scores on 1 graph)(shows minimum perceived & desired scores on 1 graph)

Another view of GapsAnother view of GapsUGA Adequacy GapsUGA Adequacy Gaps

by Questionby Question

Overall Adequacy Gap in Ascending Order

0.380.41

0.45 0.450.48

0.50.53

0.550.58 0.58

0.660.68 0.69

0.710.73

0.750.78 0.79

0.81

0.950.99

1.07

0

0.2

0.4

0.6

0.8

1

1.2

IC-8 IC-3 IC-2 IC-6 IC-5 IC-1 AS-9 IC-4 IC-7 LP-1 LP-3 LP-2 LP-4 AS-5 AS-7 AS-3 AS-4 AS-8 AS-6 AS-2 AS-1 LP-5

Question IDs

Mea

n S

co

re

Comparing gaps over timeComparing gaps over timeGT Graduates Adequacy 2003 & 2004GT Graduates Adequacy 2003 & 2004

RADAR GRAPHSRADAR GRAPHS

Radar graphs are a good way to show results of all Radar graphs are a good way to show results of all questions in one chart. This is a good visual questions in one chart. This is a good visual way to compare yourself overall with other way to compare yourself overall with other libraries.libraries.

RED RED – Scores are below your users minimum expectations– Scores are below your users minimum expectations

BLUEBLUE – shows the gap between your minimum and perceived– shows the gap between your minimum and perceived

Yellow Yellow – shows the gap between your perceived and desired– shows the gap between your perceived and desired

GREEN GREEN – shows scores above the desired expectations– shows scores above the desired expectations

LibQUAL+™ Comparisons: ARL Colleges & Universities and UGA

Core Questions Summary

ARL

UGA

BenchmarkingBenchmarking

An external focus on internal activities, functions, An external focus on internal activities, functions, or operations in order to achieve continuous or operations in order to achieve continuous improvement. improvement.

LibQUAL+ enables you to compare your library LibQUAL+ enables you to compare your library with peer institutions.with peer institutions.

LibQUAL+ enables you to identify areas of LibQUAL+ enables you to identify areas of excellence and areas for potential improvement.excellence and areas for potential improvement.

GT Benchmarking ExamplesGT Benchmarking Examples

Top Engineering SchoolsUndergraduates, Perceptions

Printed Materials

Top Engineering SchoolsUndergraduates, Adequacy

Knowledge to Answer User Questions

The survey offers a “comments box” soliciting open-The survey offers a “comments box” soliciting open-ended user views that provide a more detailed picture ended user views that provide a more detailed picture

of how users view our resources and servicesof how users view our resources and services

466 UGA participants utilized this box to give 799 466 UGA participants utilized this box to give 799 specific comments.specific comments.

231 GT participants utilized this box to give 400 231 GT participants utilized this box to give 400 specific comments. specific comments.

COMMENTS

Comments on Online Tools : Comments on Online Tools : SummarySummary

Many comments expressed confusion or dissatisfaction with doing Many comments expressed confusion or dissatisfaction with doing research online, but were not specific enough to apply to one tool such research online, but were not specific enough to apply to one tool such as GIL, GALILEO, EJL, etc. as GIL, GALILEO, EJL, etc.

Areas of difficulty/frustration expressed most frequently by survey Areas of difficulty/frustration expressed most frequently by survey participants:participants:

-the difficulty of finding journals online-the difficulty of finding journals online-too many disparate search engines available -too many disparate search engines available -online tools not user-friendly -online tools not user-friendly -searches that result in few or no hits.-searches that result in few or no hits.

Dissatisfaction with online tools seems to decrease with experience in Dissatisfaction with online tools seems to decrease with experience in searching and/or doing research, though all user groups had a high searching and/or doing research, though all user groups had a high proportion of constructive comments.proportion of constructive comments.

Comments on Online Tools:Comments on Online Tools: ““Most frustrating problem for GALILEO, etc., is the amount of drilling down Most frustrating problem for GALILEO, etc., is the amount of drilling down

to reach needed databases.” – Graduate Student, Educationto reach needed databases.” – Graduate Student, Education

“…“…sometimes GALILEO can be very confusing, particularly with all of the sometimes GALILEO can be very confusing, particularly with all of the different search engines … is there any way to put it all into ONE search?” – different search engines … is there any way to put it all into ONE search?” – Undergraduate Student, BusinessUndergraduate Student, Business

““The only major complaint I have is that searches can be very in depth and The only major complaint I have is that searches can be very in depth and complicated without assistance from a library employee at least until you complicated without assistance from a library employee at least until you have a lot of practice.” – Undergraduate Student, Journalismhave a lot of practice.” – Undergraduate Student, Journalism

““GIL is a huge problem for me. Half the time books don’t show up on GIL is a huge problem for me. Half the time books don’t show up on general or advanced searches even though there are probably over 40 general or advanced searches even though there are probably over 40 books available.” – Undergraduate Student, Social Sciencebooks available.” – Undergraduate Student, Social Science

““The online catalog search systems seem as though they were written by The online catalog search systems seem as though they were written by undergraduate computer science classes. You need a professionally written undergraduate computer science classes. You need a professionally written search system that is designed the way scientists and engineers use search system that is designed the way scientists and engineers use information. What good is a library if one can't find the information?” – information. What good is a library if one can't find the information?” – Faculty, ScienceFaculty, Science

Memorable Comments: Memorable Comments: ““The Student Learning Center is fantabulous. It’s better than a cell phone – The Student Learning Center is fantabulous. It’s better than a cell phone –

how did I live without it?!?!?! – Undergraduate, Physical Scienceshow did I live without it?!?!?! – Undergraduate, Physical Sciences

““[Suggestion:] A good cafeteria – empty stomach is also a devil’s [Suggestion:] A good cafeteria – empty stomach is also a devil’s workshop.” – Graduate Student, Ag. & Env. Sciencesworkshop.” – Graduate Student, Ag. & Env. Sciences

““Please have a drive-up renewal/fine window. That would be fabulous!” - Please have a drive-up renewal/fine window. That would be fabulous!” - Graduate Student, HumanitiesGraduate Student, Humanities

“Allow us to check out laptops like books.” – Undergraduate, Engineering

“I think the employees of the library are doing a great job. They have always been very courteous and always able to answer my questions.” Undergraduate, Engineering

“I feel that the employees could be more courteous and willing to help.” Undergraduate, Engineering

What did GT and UGA hear from their What did GT and UGA hear from their users?users?

We are not even meeting the minimum expectationsWe are not even meeting the minimum expectations for for faculty and/or graduate students in many of the information faculty and/or graduate students in many of the information control questions which includescontrol questions which includes electronic access, electronic electronic access, electronic information & print materialsinformation & print materials. (Is this because the needs are . (Is this because the needs are very specialized and/or diverse? our collections are inadequate? or very specialized and/or diverse? our collections are inadequate? or because users are unable to identify and locate what we do have?)because users are unable to identify and locate what we do have?)

All users groups, especially undergraduates, have difficulties using All users groups, especially undergraduates, have difficulties using our online tools. There is particular confusion for undergraduates our online tools. There is particular confusion for undergraduates over the terminology of GIL, GALILEO & EJL over the terminology of GIL, GALILEO & EJL

Users find it difficult to locate materials on the shelves, especially Users find it difficult to locate materials on the shelves, especially those that GIL indicates are “not checked out.” There is also some those that GIL indicates are “not checked out.” There is also some confusion on the arrangement of materials in the Library.confusion on the arrangement of materials in the Library.

DO IT AGAIN?DO IT AGAIN?

Yes or No?Yes or No?

Why?Why?

LibQUAL+ Assessment WebsitesLibQUAL+ Assessment Websites

http://old.libqual.orghttp://old.libqual.org

Here you will find information about all aspects of the surveyHere you will find information about all aspects of the survey

http://www.libs.uga.edu/assessment/index.htmlhttp://www.libs.uga.edu/assessment/index.html

This is the assessment site at UGAThis is the assessment site at UGA

http://www.library.gatech.edu/about_us/libqual/http://www.library.gatech.edu/about_us/libqual/

This is the LibQUAL+ results and analysis site at GTThis is the LibQUAL+ results and analysis site at GT

Contacts:Contacts:

Brian Mathews : Brian Mathews : [email protected]@library.gatech.edu

Caroline Killens : Caroline Killens : [email protected]@uga.edu