LibQUAL+ ® 2007: An Introduction American Library Association (ALA) Annual Conference Washington, DC June 25, 2007 MaShana Davis and Martha Kyrillidou.

LibQUAL+LibQUAL+®® 2007: An Introduction 2007: An Introduction

American Library Association (ALA)Annual Conference

Washington, DCJune 25, 2007

MaShana Davis and Martha KyrillidouAssociation of Research Libraries

old.libqual.org

old.libqual.org

OverviewOverview

• Research Foundation and Current Uses of LibQUAL+®

• Preparing to Implement the Survey

• The Survey Process: Initial Steps

• Mapping Disciplines

• Approving and Running Your Survey

• General Discussion and Q&A

old.libqual.org

LibQUAL+LibQUAL+®® Background and OverviewBackground and Overview

old.libqual.org

Reference TransactionsReference Transactions

old.libqual.org

AssessmentAssessment

“The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable—i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.”

Sarah Pritchard, Library Trends, 1996

old.libqual.org

Multiple MethodsMultiple Methodsof Listening to Customersof Listening to Customers• Transactional surveys*• Mystery shopping• New, declining, and lost-customer surveys• Focus group interviews• Customer advisory panels• Service reviews• Customer complaint, comment, and inquiry capture• Total market surveys*• Employee field reporting• Employee surveys• Service operating data capture

*A SERVQUAL-type instrument is most suitable for these methods

Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000).

Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.

old.libqual.org

World World LibQUALLibQUAL++®® Survey Survey

old.libqual.org

Premises

Three Seminal Quotations

old.libqual.org

PERCEPTIONS SERVICE “….only customers judge quality;

all other judgments are essentially

irrelevant”

Note. Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.

LibQUAL+™ Premise #1

old.libqual.org

LibQUAL+™ Premise #2

“Il est plus nécessaire d'étudier

les hommes que les livres”

—FRANÇOIS DE LA ROCHEFOUCAULD

old.libqual.org

“We only care about the things we measure.”

--Bruce Thompson, CASLIN, 2006

LibQUAL+™ Premise #3

old.libqual.org

DimensionsDimensions

2000 2001 2002 2003-200741 items 56 items 25 items 22 items

Affect of Service Affect of Service Affect of Service Affect of Service

Library as Place Library as Place Library as Place Library as Place

Reliability Reliability Personal ControlInformation Control

Provision of Physical Collections

Self-RelianceInformation Access

Access to Information

Access to Information

old.libqual.org

Interpreting Service Quality Data

Three Interpretation Frameworks

old.libqual.org

Benchmarking Against Peer Institutions

--1,000,000 Users; 1,000 Institutions!

NORMS! NORMS! NORMS!

Interpretation Framework #1

old.libqual.org

Score Norms

• Norm Conversion Tables facilitate the interpretation of observed scores using norms created for a large and representative sample.

• LibQUAL+™ norms have been created at both the individual and institutional level

old.libqual.org

Institutional Norms for PerceivedMeans on 25 Core Questions

Note: Thompson, B. LibQUAL+ Spring 2002 Selected Norms, (2002).

old.libqual.org

Benchmarking Against Self, Longitudinally

“Nobody is more like me than me!”--Anonymous

Interpretation Framework #2

old.libqual.org

Interpreting Perceived Scores Against Minimally-Acceptable and Desired

Service Levels (i.e., “Zones of Tolerance”)

Interpretation Framework #3

old.libqual.org

old.libqual.org

old.libqual.org

Survey Structure Survey Structure (Detail View)(Detail View)

old.libqual.org

Rapid Growth in Other AreasRapid Growth in Other Areas• Languages

– Afrikaans– American English– British English– Danish– Dutch – Finnish– French– German– Norwegian– Swedish

• Consortia*Each may create 5 local questions to add

to their survey

• Countries– Australia, Canada, Denmark, Egypt,

Finland, France, Hong Kong, New Zealand, the Netherlands, Norway, South Africa, Sweden, Switzerland, UAE, U.K., U.S.

• Types of Institutions– Academic Health Sciences– Academic Law– Academic Military– College or University– Community College– Electronic– European Business– European Parliament– Family History– Research Centers (FFRDC) Libraries– High School (2007)– Hospital– National Health Service England– Natural Resources– New York Public– Public– Smithsonian– State– University/TAFE

old.libqual.org

250

218

307308

204

164

13

43

152,111

176,360

151,460

113,480

78,863

4,407

20,416

128,958

0

50

100

150

200

250

300

350

2000 2001 2002 2003 2004 2005 2006 2007

Survey Year

Number ofInstitutions

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,000

180,000

200,000

Number ofResponses

Number of Institutions

Number of Responses

LibQUALLibQUAL++®® Participants Participants

* 2007 data reflects Session I data only

old.libqual.org

LibQUALLibQUAL++®® First Year Participants First Year Participants

* 2007 data reflects Session I data only

old.libqual.org

LibQUALLibQUAL++®® Surveys by Type Surveys by Type  Year

Type 2000 2001 2002 2003 2004 2005 2006 2007

Academic Health Sciences   1 35 23 13 13 10 8

Academic Law       1 25 10 6 2

Academic Military       6   1   1

College or University 13 41 111 244 150 201 226 177

Canadian Government               6

Community College     16 29 3 15 27 18

Electronic           1    

European Business         5   16  

Family History         1   2  

Hospital         10 1 1  

National Health Services England             10  

Natural Resources           4    

New York Public     1          

Public       4 1 1 3 2

Research Centers (FFRDC) Libraries           5 2 1

Smithsonian     1   1      

State       1   1 3 2

University/TAFE           2 1 1

* 2007 data reflects Session I data only

old.libqual.org

LibQUALLibQUAL++®® Languages Languages

American English

Dutch EnglishFrench Canadian DutchSwedish

Swedish(British English)

Afrikaans

DanishFinnishGerman Norwegian

British English

Continental French

old.libqual.org

LibQUALLibQUAL++®® Surveys by Language Surveys by Language  Year

Language 2000 2001 2002 2003 2004 2005 2006 2007

Afrikaans           4 1  

American English 13 42 164 285 176 207 236 179

British English       20 22 31 50 18

Danish             1  

Dutch         1   2  

Dutch English       1 1   2  

Finnish             1  

French Canadian       2 1 4   20

French Continental         1   1  

German             1  

Norwegian             1  

Swedish         5 2 1 1

Swedish British English             1  

Swedish English (A.E.)         2 1    

* 2007 data reflects Session I data only

old.libqual.org

LibQUALLibQUAL++®® Surveys by Consortia Surveys by Consortia  Year

Consortium 2000 2001 2002 2003 2004 2005 2006 2007

AAHSL   1 35 21 14 12 7 8

Alabama Academic (NAAL)       10 1 8 2 11

AJCU - Academic Libraries         20    21  

AJCU - Law Libraries             1   

Connecticut State University Library System (CONSULS)              5  

Consortium of Church Libraries and Archives              7  

CUC Group (formerly CES)           6  8  1

CUNY Libraries            19    

Department of Justice Canada Libraries         6     17  

European Business Schools Librarians Group                

Georgia Consortium              19 1

Harrisburg Area Community Colleges              5  

Hospital/MLA          7      

Keystone Library Network              15  

LibQUAL Canada               63

* 2007 data reflects Session I data only

old.libqual.org

LibQUALLibQUAL++®® Surveys by Consortia Surveys by Consortia(cont’d)(cont’d)

Year

Consortium 2000 2001 2002 2003 2004 2005 2006 2007

Maine URSUS Libraries                13

Massachusetts LSTA              5  

MCCLPHEI               14 

MERLN (Military Education & Research Library Network)        6        

National Health Service England              10  

New England Law Library Consortium (NELLCO)            8    

North Carolina Community Colleges             15   

NY3Rs College and University Libraries       71    2 1 1 

NY3Rs Public Libraries       5         

Oberlin Libraries Group        12 9 13 2 1

OhioLINK      57 45 1 14 2 6

Research Centers (FFRDC) Libraries        

SCONUL        20 17 16 20 14

State University Libraries of Florida          6 2 1 1

University of Wisconsin System          14      

VALE           12  1   

* 2007 data reflects Session I data only

old.libqual.org

Participating Libraries by CountryParticipating Libraries by Country  Year

Country 2000 2001 2002 2003 2004 2005 2006 2007

Australia         1 6 2 1

Canada 1 3 4 8 10 15 11 63

Denmark         1   2  

Egypt         1   2 1

Finland             2  

France         1   2 1

Ireland         1 1 2 1

Mexico               1

Netherlands       1 1   5  

New Zealand             1 1

Norway             2  

South Africa           12 8 2

Sweden         3 4 4 2

Switzerland         2   2  

U.A.E.         1      

U.K.       20 17 16 33 14

* 2007 data reflects Session I data only

old.libqual.org

Surveys by Session: 2004-2007Surveys by Session: 2004-2007

Year Session 1 Session 2

2004 202 2

2005 199 56

2006 205 93

2007 218

old.libqual.org

Preparing for the Survey Preparing for the Survey ImplementationImplementation

old.libqual.org

Why is Your Library Participating Why is Your Library Participating in in LibQUAL+LibQUAL+® ® ??

• Institutional goals:

– what do you want to get out of the survey?

• Survey requirements:

– people, supplies, technology

old.libqual.org

Get Permission from your Get Permission from your Institutional Review BoardInstitutional Review Board

• Local group that approves human subject research

• May also be called

– Committee for Human Subject Research

– Committee for the Protection of Human Subjects

• Not all institutions need to obtain permission

old.libqual.org

Get Permission from yourGet Permission from yourInstitutional Review BoardInstitutional Review Board

• If permission is required:

– Seek well in advance of survey

– Supply a copy of the survey, if requested

– Inform them that results will be shared among participants

• No need to inform LibQUAL+® of decision

old.libqual.org

Determine Whom to SurveyDetermine Whom to Survey

• Random sample

or

• Entire population

old.libqual.org

If You Sample…If You Sample…

• Recommendations: – At least 1,200 random email addresses for

each user group– Separate sample groups for

undergraduates, graduates, faculty, and staff

– Think about survey fatigue if you want to repeat the survey regularly

• Keep notes on your methodology as you will be asked to describe it in the Post Hoc Questionnaire

old.libqual.org

If You Survey the If You Survey the Entire Population….Entire Population….

• Recommendations:

– Think about survey fatigue if you want to repeat the survey regularly

• Keep notes on your methodology as you will be asked to describe it in the Post Hoc Questionnaire

old.libqual.org

Obtain Email AddressesObtain Email Addresses

• Typical sources include:

– Campus computing office

– Campus administrative records office

– Library patron database

old.libqual.org

Marketing Your SurveyMarketing Your Survey

– Place ads in campus newspaper

– Write article for library newsletter

– Post flyers around campus

– Present at faculty meetings & student orientations

– Create a survey Web site and feature on library’s home page

– Take the survey to where users are: dining halls, study rooms, dorms

old.libqual.org

Become Familiar with Become Familiar with LibQUAL+LibQUAL+®® Resources Resources

• Management Centerhttp://www.libqual.org/Manage/index.cfm

• LibQUAL+® Procedures Manualhttp://www.libqual.org/Manual/index.cfm – Updated Version – September 2006– Much more detail

• Discussion [email protected]

old.libqual.org

The The LibQUAL+LibQUAL+®® Management CenterManagement Center

old.libqual.org

The Survey Process: Initial StepsThe Survey Process: Initial Steps

old.libqual.org

Survey Process: Manage Your SurveySurvey Process: Manage Your Survey

Eight Sections:

• Preferences

• Customization

• Preview

• Representativeness

• Monitor Survey Progress

• Incentive Winners

• Post Hoc

• Evaluation

*Sections must be completed in order

old.libqual.org

PreferencesPreferences

Fields marked with * are required

old.libqual.org

Customization: Optional/Local Customization: Optional/Local QuestionsQuestions

old.libqual.org

Customization: DisciplinesCustomization: Disciplines

• Results notebooks summarize findings by user group and provide a chart for, both, standard and custom disciplines

• Standard disciplines (based on your institution type, i.e., College/University)

• Customized disciplines

– Recommend no more than 16 disciplines, if possible

old.libqual.org

The Standard Disciplines – The Standard Disciplines – College/UniversityCollege/University• Agriculture/Environmental

Studies• Architecture• Business• Communications/

Journalism• Education• Engineering/Computer

Science• General Studies• Health Sciences

• Humanities• Law• Military/Naval Science• Other• Performing & Fine Arts• Science/Math• Social Sciences/

Psychology• Undecided

old.libqual.org

Customized DisciplinesCustomized Disciplines

• Use your local terminology to map to the standard disciplines

• Cautions:

– Need to provide representativeness data for each discipline

– Too many choices present challenges to users

old.libqual.org

Standard Disciplines: Standard Disciplines: An Academic LibraryAn Academic Library

old.libqual.org

Customized Disciplines: Customized Disciplines: An Academic LibraryAn Academic Library

old.libqual.org

Customization: Branch LibrariesCustomization: Branch Libraries

The library you use most often.

old.libqual.org

Approving and Running Approving and Running Your SurveyYour Survey

old.libqual.org

Previewing and ApprovingPreviewing and ApprovingYour SurveyYour SurveyPreview• Complete at least one full run

of your preview survey• Test in different settings,

using different platforms and Web browsers

• Get library staff involved in testing

Approved• Can no longer make changes• Live survey URL will appear at

the top of the “Manage Your Survey” page marked as Survey URL

old.libqual.org

Reviewing CommentsReviewing Comments

• About one-half of users include comments on their survey

• Download via the Monitor Survey Progress page (Excel file or text file)http://old.libqual.org/Manage/Results/Comments/index.cfm

old.libqual.org

Sending RemindersSending Reminders

• 3-5 reminders to sample populations

• Thank respondents who have completed the survey

• Boost up marketing efforts around campus– More fliers, table tents, ads in campus

newspaper– Get professors and other staff involved– Increase number or types of incentives

* More examples of marketing efforts featured at the Share Fair

old.libqual.org

Completing Your Completing Your Representativeness QuestionnaireRepresentativeness Questionnaire• Determines how your institutional profile compares to

your survey data• Requires the following information:

– # of individuals per user group– # of individuals within each discipline– # of males and females– Library Statistics

• Volumes added during the year – Gross/Total (including e-books)• Total number of current serials received (including electronic serials)• Total library expenditures (U.S. dollars)• Personnel – professional staff, full-time equivalent (FTE)• Personnel – support staff, full-time equivalent (FTE)

• Must be complete BEFORE closing survey

old.libqual.org

Completing Your Completing Your Representativeness QuestionnaireRepresentativeness Questionnaire

Representativeness Representativeness CompletedCompleted

Representativeness Representativeness NOT CompletedNOT Completed

old.libqual.org

Closing Your SurveyClosing Your Survey

• We recommend a survey run of at three weeks

• Once you close your survey:– Retrieve list of incentive winners– Fill out Post Hoc and Evaluation

Questionnaires

old.libqual.org

Completing Post Hoc and Evaluation Completing Post Hoc and Evaluation QuestionnairesQuestionnaires

Post Hoc Questionnaire• Information about your

survey– Sample size– # of e-mails sent– #of invalid e-mail

addresses– Incentives offered– Marketing techniques– Etc.

Evaluation Questionnaire• Feedback about your

LibQUAL+® experience• All survey liaisons and

assistants are encouraged to complete this questionnaire

old.libqual.org

General Discussion General Discussion and Q&Aand Q&A

old.libqual.org

Summary and ClosureSummary and Closure

old.libqual.org

LibQUAL+LibQUAL+®® Resources Resources

• LibQUAL+® Web site:http://old.libqual.org

• Publications:http://old.libqual.org/publications

• Events and Training:http://old.libqual.org/events

• Gap Theory/Radar Graph Introduction:http://old.libqual.org/Information/Tools/libqualpresentation.cfm

• LibQUAL+® Procedures Manual:http://old.libqual.org/Publications/index.cfm

old.libqual.org

The The LibQUAL+LibQUAL+®® Team Team• MaShana Davis

Technical Communications [email protected]

• Yolanda GlassAdministrative [email protected]

• Kristina JusthCustomer Relations [email protected]

• Martha KyrillidouDirector, Statistics and Service Quality [email protected]

• Khyati NayakSenior Applications [email protected]

• Gary RoebuckTechnical Operations [email protected]