1 GreekAcadLibs17 JED 2008 © J.E. Davies What does success look like? Evidence–based assessment of the academic library Dr. J. Eric Davies Honorary Visiting.

1GreekAcadLibs17 JED 2008 © J.E. Davies

What does success look like? What does success look like? Evidence–based assessment of the Evidence–based assessment of the

academic libraryacademic library

Dr. J. Eric Davies

Honorary Visiting Research Fellow

Loughborough University

17th Conference Greek Academic Libraries


Outline:Outline:

• Management environmentManagement environment• Success factors, mission and strategySuccess factors, mission and strategy• Performance evaluation and enhancementPerformance evaluation and enhancement• Evidence typesEvidence types• Culture and competencesCulture and competences• Future trends speculations Future trends speculations



Management environment:Management environment:

Change –Change –

• TechnologyTechnology

• Scholarly Communication and InformationScholarly Communication and Information

• Attitudes/Expectations/DemandsAttitudes/Expectations/Demands

• Services, Information Skills and SupportServices, Information Skills and Support

• Finance/Resources/AccountabilityFinance/Resources/Accountability

• Innovation/DevelopmentInnovation/Development



Success factors, mission and strategy:Success factors, mission and strategy:

• Success Framework/CriteriaSuccess Framework/Criteria

• Mission/Vision/ValuesMission/Vision/Values

• Aims/Objectives/StrategiesAims/Objectives/Strategies

• Service goals/TargetsService goals/Targets

• Performance IndicatorsPerformance Indicators

• Evidence Based ManagementEvidence Based Management



Framework for Performance Measurement:-Framework for Performance Measurement:-

Integration with institutional goals Integration with institutional goals

User satisfaction User satisfaction

Effectiveness (delivery) Effectiveness (delivery)

Efficiency Efficiency

EconomyEconomyFollett Report – academic libraries –1993Follett Report – academic libraries –1993



Special Libraries Association –Special Libraries Association –

For special librarians, evidence-based practice For special librarians, evidence-based practice

refers to consciously and consistently making refers to consciously and consistently making

professional-level decisions that are based on professional-level decisions that are based on

the strongest evidence of what would work best the strongest evidence of what would work best

for our clients.for our clients.



Types of evidence –Types of evidence –

• Performance EvidencePerformance Evidence

• Research EvidenceResearch Evidence



Systematically gathered, structured and Systematically gathered, structured and contextualised information relating to local contextualised information relating to local operations and needsoperations and needs

Performance EvidencePerformance Evidence



Performance Evidence –Performance Evidence –

Services Services

ProcessesProcesses

QuantitativeQuantitative

QualitativeQualitative



Inputs Inputs

Quantitative DataQuantitative Data OutputsOutputs

Qualitative Evidence Qualitative Evidence OutcomesOutcomes

User ResponsesUser Responses

ImpactsImpacts

Performance



User Responses –User Responses –

• SurveysSurveys

• Focus groupsFocus groups

• Open DaysOpen Days

• LibQUALLibQUAL++TMTM

• Rodski/InsyncRodski/Insync



User Surveys –User Surveys –

• User Practice [frequency of visits, types of use]User Practice [frequency of visits, types of use]• User Capabilities [ IT use, vision, mobility]User Capabilities [ IT use, vision, mobility]• User Motivations [reasons for use/non-use]User Motivations [reasons for use/non-use]• User Preferences [service development]User Preferences [service development]• User Needs/Wants [service priorities]User Needs/Wants [service priorities]• User Opinions [perceptions of service]User Opinions [perceptions of service]• User Satisfaction [with various features]User Satisfaction [with various features]



0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Feb-02

Mar-03

Feb-04

Mar-05

Feb-02

Mar-03

Feb-04

Mar-05

Sa

tisfa

ctio

n w

ith s

taff

Sa

tisfa

ctio

n w

ithse

rvic

es/

faci

litie

s

Excellent

Good

Fair

Poor

British Library reader satisfaction survey ratings


LibQUAL+LibQUAL+TMTM

Developed by ARL/Texas A&M UnivDeveloped by ARL/Texas A&M Univ

Based on SERVQUAL - Gap Analysis Based on SERVQUAL - Gap Analysis

Minimum, Desired, Perceived user view of quality Minimum, Desired, Perceived user view of quality

relating to -relating to -

•Access to InformationAccess to Information

•Affect of ServiceAffect of Service

•Library as PlaceLibrary as Place

•Personal ControlPersonal Control



4

5

6

7

8

9

AI-1

AI-2

AI-3

AI-4

AI-5

AS-1

AS-2

AS-3

AS-4

AS-5

AS-6

AS-7AS-8AS-9

LP-1

LP-2

LP-3

LP-4

LP-5

PC-1

PC-2

PC-3

PC-4

PC-5

PC-6

Perceived greater than desired Perceived less than desired

Perceived greater than minimum perceived less than minimum

Example of data display of LibQUAL+TM

Results


XXX

4

5

6

7

8

9

AI-1

AI-2

AI-3

AI-4

AI-5

AS-1

AS-2

AS-3

AS-4

AS-5

AS-6

AS-7

AS-8

AS-9

LP-1

LP-2

LP-3

LP-4

LP-5

PC-1

PC-2

PC-3

Perceived greater than desired Perceived less than desired

Perceived greater than minimum perceived less than minimum

Example of data display of LibQUAL+TM

Results


Example of Gap Analysis from LibQUAL+TM Results

0

1

2

3

4

5

6

7

8

9

Access toInformation

Affect of Service Library asPlace

Personalcontrol

Range ofminimum todesired

Range ofminimum toperceived


The amount of information and a manager’s ability to use it have The amount of information and a manager’s ability to use it have

increased explosively not only for internal processes but also for increased explosively not only for internal processes but also for

the engagement of customers. The more a company knows about the engagement of customers. The more a company knows about

them, the better able it is to create offerings they want, to target them, the better able it is to create offerings they want, to target

them with messages that get a response, and to extract the value them with messages that get a response, and to extract the value

that an offering gives them. The holy grail of deep customer insight that an offering gives them. The holy grail of deep customer insight

– more granular segmentation, low-cost experimentation, and mass – more granular segmentation, low-cost experimentation, and mass

customization – becomes increasingly accessible through customization – becomes increasingly accessible through

technological innovations in data collection and processing in technological innovations in data collection and processing in

manufacturing.manufacturing. James M. Manyika, Roger P. Roberts and Kara L. Sprague ~ James M. Manyika, Roger P. Roberts and Kara L. Sprague ~

McKinsey Q.(1) 2008. McKinsey Q.(1) 2008.



Service Service

DomainDomain Community ProfilingCommunity Profiling

DemographicsDemographics

User DataUser Data

Lapsed, Non-Lapsed, Non-User, Potential User DataUser, Potential User Data

Information EnvironmentInformation Environment

Performance



Comparative Benchmarking –Comparative Benchmarking –

Evaluate the level of performance of Evaluate the level of performance of various services within an institutionvarious services within an institution

Compare performance over timeCompare performance over time Compare with other institutionsCompare with other institutions Compare with recognised standardsCompare with recognised standards




STRATEGIC STRATEGIC

macro performance metrics macro performance metrics and qualitative evidenceand qualitative evidence

PROCESS PROCESS

detail cost performance datadetail cost performance data




To make changes that lead to quantum and To make changes that lead to quantum and continuous improvements in products, continuous improvements in products, processes and services that result in total processes and services that result in total customer satisfaction and competitive customer satisfaction and competitive advantage.advantage.

[Motorola + D.E.C. + Xerox][Motorola + D.E.C. + Xerox]



Percentage of requests met in 7,15 & 30 days

0%

20%

40%

60%

80%

100%

98/9 99/0 00/1 01/2 02/3 03/4

XXX 30

XXX15

XXX 7

Std 30

Std 15

Std 7


0

1

2

3

4

5

6

7

8

9

10

1997-98 1998-99 1999-00 2000-01 2001-02 2002-03 2003-04

XXXXX

Group

CURL

SCONUL

University Library - FTE students per seat


0

5

10

15

20

25

30

35

40

45

1991-92 1992-93 1993-94 1994-95 1995-96 1996-97 1997-98 1998-99 1999-00 2000-01

£

Library X Comparators New SCONUL

University Library - Book expenditure per FTE student


Average Weekly Opening Hours

0

10

20

30

40

50

60

70

80

90

94-5 95-6 96-7 97-8 98-9 99-0 00-1 01-2 02-3 03-4 04-5

xxxxxxxxx

Benchmark

Old

Old + CURL

SCONUL


Re-shelving and tidying performance

Site A

Site B

Site C

Site D

Site E

Site F

Site G

0%

20%

40%

60%

80%

100%

0 2 4 6 8 10 12 14 16 18 20 22 24

Number of hours


Research based intelligence and macro data Research based intelligence and macro data that informs service design and decisionsthat informs service design and decisions

Research EvidenceResearch Evidence









Performance Evidence Culture –Performance Evidence Culture –

Service design and delivery are informed Service design and delivery are informed by appropriate internal and external by appropriate internal and external evidenceevidence ‘ ‘Think Evidence’ attitude permeates the Think Evidence’ attitude permeates the organisationorganisation Skills, knowledge, attitudes in using Skills, knowledge, attitudes in using evidence are embedded at ALL levels of evidence are embedded at ALL levels of the organisationthe organisation



What will success look like – What will success look like – tomorrow??? tomorrow???

• Optimal Management Optimal Management

• Maximised Value AddedMaximised Value Added

• Academic Culture/EnvironmentAcademic Culture/Environment



How will we measure success – How will we measure success – tomorrow??? tomorrow??? • Contingent valuation analysisContingent valuation analysis• Economic impact studiesEconomic impact studies• Critical incident studies Critical incident studies • Scenario planning Scenario planning • Lean managementLean management• Six sigmaSix sigma



‘‘Management in particular is not about the preservation Management in particular is not about the preservation of the status quo, it is about maintaining the highest of the status quo, it is about maintaining the highest rate of change that the organization and the people rate of change that the organization and the people within it can stand.’within it can stand.’

‘‘Whether change is comfortable or not, it is inevitable.’Whether change is comfortable or not, it is inevitable.’

[ Harvey-Jones, J. [ Harvey-Jones, J. Making it happen: Reflections on Making it happen: Reflections on leadershipleadership. London, Collins, 1988. ]. London, Collins, 1988. ]



Thank you for your attention.Thank you for your attention.

Questions?Questions?




1 GreekAcadLibs17 JED 2008 © J.E. Davies What does success look like? Evidence–based assessment of the academic library Dr. J. Eric Davies Honorary Visiting.

Documents

greekacadlibs17 jed

management evidence

strongest evidence

davies success factors

academic library

davies management environment

davies outline

davies framework