Strategic collection management through statistical analysis

Strategic Collection Management through Statistical Analysis

Stephanie H. Wical

University of Wisconsin – Eau Claire

&

Hans Kishel

University of Wisconsin – Eau Claire

NASIG 2012

Methodology

• Identified academic libraries in Wisconsin using a directory maintained by Wisconsin Association of Academic Libraries.

• Surveyed academic librarians who were believed to have some role (if not sole responsibility) for electronic resource management.

• Interviewed academic librarians (who were a subset of the group who responded to the survey).

NASIG 2012

Demographics of the Group

The group included libraries in public and private colleges and universities, technical colleges, two-year colleges and for-profit colleges and universities in the state of Wisconsin or academic libraries that provide library service to residents of the state of Wisconsin.

NASIG 2012

Response Rate

• 139 surveys distributed via e-mail• 62 completed survey responses• 28 completed phone interviews

45% Response Rate for the Survey as of May 25, 2012

NASIG 2012

Survey Question: What types of library electronic resources statistics--if any--do you currently collect?

(Choose all that apply.)

Answer Responses PercentageSearches 54 87%Sessions 46 74%

Full Text Downloads 41 66%Cost-per-use 25 40%

IP Addresses that access resources 3 5%Simultaneous Users 9 15%

Turn-aways 27 44%

Times authors at your college or university are cited in resources 1 2%Times authors at your college or university cite resources 2 3%

Scholarly Impact 1 2%Not sure 3 5%

None 1 2%Other (Please specify.) 6 10%

N=62

NASIG 2012

Survey Question: What measure or measures do you consider when evaluating electronic resources in your

library? (Choose all that apply.)

Answer Response PercentageSearches 53 85%Sessions 39 63%

Full Text Downloads 38 61%Cost-per-use 37 60%IP Addresses 1 2%

Simultaneous Users 19 31%Turn-aways 21 34%

Times authors at your college or university are cited in resources 3 5%Times authors at your college or university cite resources 3 5%

Scholarly Impact 20 32%Not sure 2 3%

None 1 2%Other (Please specify.) 7 11%

N=62

NASIG 2012

A Comparison of Collected and Considered

Searc

hes

Sessi

ons

Full T

ext D

ownloads

Cost-per-

use

IP Addresses

that

access

reso

urces

Simulta

neous U

sers

Turn

-away

s

Times

authors

at yo

ur colle

ge or u

niversi

ty are

cited

in re

source

s

Times

authors

at yo

ur colle

ge or u

niversi

ty cit

e reso

urces

Scholar

ly Im

pact

Not sure

None

Other (Plea

se sp

ecify.

)0

10

20

30

40

50

60

ConsideredCollected

N=62

NASIG 2012

Searches, Sessions, Full Text Downloads and Turnaways

NASIG 2012

Survey Question: How frequently do you consider your measure or measures?

Never2%

Once a year39%

During the fall and spring semesters16%

Quarterly6%

Monthly16%

Not sure5%

Other (Please specify.)16%

N=62

NASIG 2012

Interview Question: Do you look at cost-per-use for electronic resources?

N=28

Yes68%

No32%

NASIG 2012

IP Addresses That Access Resources

This measure might be important down the road as departments outside of the library might have to share costs for resources.

NASIG 2012

Times Cited or Times Citing

Collecting this type of information was not a high priority for any of the librarians we talked to.

NASIG 2012

Scholarly Impact

• Scholarly impact is not something one routinely tracks, but it is something that is valuable when considering adding a new resource.

Image from Experimental Physiology at http://ep.physoc.org/site/misc/author.xhtml

NASIG 2012

Interview Question: What measure or measures do you consider when evaluating electronic resources in your library?

Sessi

ons

Searc

hesFT

-DL

Cost-per-

use

Turn

AwayPric

e

Duplication

Page V

iews

Click Th

rough

Releva

nce to pro

grams

Overla

p

Ease

of use

Curricu

lum

Coverag

e

Review

s

Authority

Platform

Quality

Usage S

tats

0%

10%

20%

30%

40%

50%

60%

70%

N=28

NASIG 2012

Interview Question: Why do you use the measure or measures you use to evaluate electronic resources?

Budget We have always done it that way / It is what

we get

To get idea of what students are using

Renewal To be able to compare

0%

5%

10%

15%

20%

25%

30%

35%

N=28

NASIG 2012

Interview Question: Do you feel that the measure or measures you use are adequate or do you feel that they do not provide relevant information?

Yes50%

So-So29%

No21%

N=28

NASIG 2012

People who were not satisfied with usage statistics indicated:

• Not all of them are counter compliant.• Could always use more data.• Sometimes usage statistics do not account for a lot

of variables.• It would be nice to find a measure that can be looked

at across vendors.• They are not relevant at all; statistics are a joke.• We are drowning in data.

NASIG 2012

Where one library may see value in a set of statistics, another library may need something more.

---Schufreider & Romaine (2008)

NASIG 2012

Survey Question: What tool or tools do you use to gather statistics for evaluating electronic journals, databases and/or

electronic books? (Choose all that apply.)

Answers Number Using Tool PercentagesSFX Ustat 5 8%

ERMes (developed at UW- La Crosse) 8 13%Scholarly Stats 1 2%

Serials Solution 360 Link 4 7%Serials Solutions 360 Resource Manager 5 8%

EBSCO ERM Essentials 4 7%EBSCO LinkSource 7 11%

Ex Libris’ Verde 1 2%Innovative ERM 1 2%

Open ERM 1 2%OCLC Worldcat Link Manager 2 3%

CORAL ERM 1 2%Excel spreadsheet or other spreadsheet software 40 66%

Access database or other database software 10 16%Not sure 10 16%

None 2 3%Other (Please specify.) 15 25%

N=61

NASIG 2012

Finding

66% are using Excel in addition to another tool

Q6 What tool or tools do you use to gather statistics for evaluating electronic journals, databases and/or

electronic books? Excel spreadsheet or other spreadsheet software

SFX Ustat

ERMes (develop

ed at UW- La Crosse)

Scholarly Stats

Serials Solution 360 Link

Serials Solutions

360 Resource Manager

EBSCO ERM

Essentials

EBSCO LinkSourc

eEx Libris’

VerdeInnovativ

e ERMOpen ERM

OCLC Worldcat

Link Manager

CORAL ERM Not sure

Other (Please specify.)

Excel spreadsheet or other spreadsheet software 4 7 1 3 2 4 6 1 1 1 1 1 1 7

N=61

NASIG 2012

Survey Question: Are you able to compare statistics across vendors and platforms?

Yes60%

No11%

Not sure16%

Other (Please spec-ify.)13%

N=62

NASIG 2012

Survey Question: Have you taken advantage of the Standardized Usage Statistics Harvesting Initiative (SUSHI)

protocol to obtain usage reports?

Yes7%

No57%

Plan to do so in the next two years

13%

Not sure16%

Other (Please specify.)7%

N=61

NASIG 2012

Survey Question: How important are usage statistics in the decision to renew or cancel electronic resources?

Very unimportant8% Unimportant

2%Neither unimportant nor important (neu-

tral)16%

Important39%

Very important35%

N=62

NASIG 2012

Survey Question: Have you ever canceled an electronic resource because it had low use?

Yes81%

No8%

Not sure11% N=62

NASIG 2012

“How important are usage statistics in the decision to renew or cancel electronic resources?”

&“Have you ever canceled an electronic resource

because it had low use?”

How important are usage statistics in the decision to renew or cancel electronic

resources?Have you ever canceled an electronic resource because it had low use? Grand Total

Yes NoNot sure

Very unimportant 3 2 5

Unimportant 1 1

Neither unimportant nor important (neutral) 6 3 1 10

Important 20 1 3 24

Very important 21 1 22

Grand Total 50 5 7 N=62

NASIG 2012

What do you collect and how important is it?

Q3 What types of library electronic resources statistics--if any--do you currently collect? (Choose all that apply.)

Q14 How important are usage statistics in the decision to renew or cancel electronic resources? Searches Sessions

Full Text Download

sCost-per-

use

IP Addresses

that access

resources

Simultaneous Users

Turn-aways

Times authors at

your college or university are cited

in resources

Times authors at

your college or university

cite resources

Scholarly Impact Not sure None

Other (Please specify.)

Very unimportant 4 4 4 4 4 1 1 1Unimportant 1 Neither unimportant nor important (neutral) 8 7 6 2 1 3 1 Important 21 19 16 8 2 7 11 1 1 1 3Very important 21 16 15 11 1 1 9 1 2

Grand Total 54 46 41 25 3 9 27 1 2 1 3 1 6

NASIG 2012

Using Statistics

What will you do with usage statistics?– Make changes and improvements– Request additional resources based

upon data

NASIG 2012

Interview Question: What do you think that you should be doing with electronic resource usage statistics?

Marketing12% What is getting used

12%

Bang for the Buck15%

More indepth12%

Communication19%

Renewal/informed decisions

19%

Better organized / SUSHI12%

N=28

NASIG 2012

Survey Question: Do you plan to collect additional statistics in the next two years?

Yes37%

No10%

Maybe; it will depend on cir-

cumstances (e.g. budget or

staffing)42%

Not sure11%

N=62

NASIG 2012

Sharing Usage Statistics

• Who needs to know about usage statistics?– Identify stakeholders– What will the reports look like?– What is the right amount of information?– Should the reports look different for

different people?– Should an executive summary be

written for an administrator?

NASIG 2012

Survey Question: Are usage statistics for library electronic resources reported outside the library?

N=62

Yes50%

No24%

Not sure26%

NASIG 2012

Survey Question: Who receives regular electronic resources statistical reports from the person or people who collect them?

(Choose all that apply.)

Answers Responses PercentagesElectronic Resources Librarian 10 16%Periodicals Librarian 7 11%Technical Services Librarian 5 8%Director / Dean 30 48%Administrator who supervises the director or dean of the library 7 11%Reference Librarian 10 16%Systems Librarian 5 8%Everyone in the library 13 21%Everyone on the e-resources team, work group or task force 4 6%Anyone who reads the annual report 5 8%Anyone who reads the library newsletter 0 0%Anyone who reads the library blog, Facebook page, or twitter feed 0 0%No one 7 11%Not sure 4 6%Other (Please specify.) 16 26%

N=62

NASIG 2012

Survey Question: Who collects electronic resources statistical reports at your library? (Choose all that apply.)

Answers Responses PercentagesElectronic Resources Librarian 17 27%Periodicals Librarian 7 11%Technical Services Librarian 6 10%Director / Dean 13 21%Reference Librarian 6 10%Systems Librarian 8 13%Paraprofessional 13 21%Student Workers 2 3%We don’t collect them at our library---they are collected on a system or consortial level 10 16%Our vendor collects them for us 11 18%We do not collect them at all 1 2%Not sure 0 0%Other (Please specify.) 10 16%

N=62

NASIG 2012

Survey Question: Is collecting electronic resources statistical data something that is written into one or

more job descriptions?

N=62

Yes37%

No44%

Not sure18%

Other (Please spec-ify.)2%

NASIG 2012

Interview Question: What would you consider a best practice for collection management statistics for electronic resources?

Answer # of responses %No Idea 7 25%

ERM 2 7%SUSHI 5 18%

Collect Regularly 1 4%Check them to make sure they are right 1 4%

Big Picture thinking 2 7%Compare over time 3 11%

Feedback 2 7%Review Regularly 1 4%How students use 1 4%

Cost per use 1 4%COUNTER 2 7%Use them 2 7%

N=28

NASIG 2012

Interview Question: How do you prioritize uses of the electronic resources usage statistics you collect? Or what is most

important?

Answers Number of Responses PercentagesWhat getting used 9 32%

Cost per use 1 4%Renewal 6 21%

Annual Report 5 18%Turnaways 1 4%

FT-DL 4 14%Search 5 18%

Course integration 2 7%Sessions 1 4%Budget 4 14%

Faculty Requests 1 4%Vendor Data 2 7%ACRL Reports 1 4%

Don't 1 4%Advertising 1 4%

Bang for Buck 2 7%Look for Changes/Problems 2 7%

N=28

NASIG 2012

Assessment Process

NASIG 2012

Survey Question: Has there been any attempt at your library to incorporate usage data into assessment of library services?

Yes48%

No35%

Not sure16%

N=62

NASIG 2012

Interview Question: How do you determine if usage statistics are evaluated and consulted successfully?

Don't59%

If there is a problem then something is

wrong15%

Feedback26%

N=28

NASIG 2012

Statistics can play a more progressively important role in determining the return on investment with library dollars.

---Beals & Lesher (2010)

NASIG 2012

80/20 Distribution

Thomas Nisonger (2008) states, “The basic 80/20 pattern provides a valid approach to operationalizing the core journal concept and is applicable to collection management decision making” (p. 78).

Although Nisonger (2008) admits that the percentages do not match the 80/20 rule exactly, should we make sure that the majority of our budget is going into resources that get the majority of use?

NASIG 2012

Quantitative Data

– Cost-per-use– Number of disciplines– Historical usage data (including five year trends)– Interlibrary loan data– Impact factors

Source: Hulbert, Roach & Julian (2011)

NASIG 2012

Qualitative Data

– Experience or the gut feeling– User feedback– Reviews– The dust factor for print resources

Source: Hulbert, Roach & Julian (2011)

NASIG 2012

Decisions and planning up front are often the most time-consuming part of the process.

--Hulbert, Roach & Julian, 2011

NASIG 2012

Case Study INancy Beals of Wayne State University created a “Stats Master” spreadsheet and training program for selectors (Beals & Romaine, 2010).

NASIG 2012

Case Study IILinda Hulbert & Dani Roach of University of St. Thomas Libraries developed an “Electronic Resources Review Checklist” and a way to “connect the silos” to contribute to the university’s overall culture of assessment (Hulbert, Roach & Julian, 2011)

NASIG 2012

Case Study IIIMary Ann Trail and Kerry Chang-FitzGibbon describe how the Library of the Richard Stockton College of New Jersey addressed a $29,000 shortfall using an overlap analysis report (Trail, Chang-FitzGibbon & Wishnetsky (2012).

NASIG 2012

Case Study IVBryan Vogh and Hans Kishel (2011)at the University of Wisconsin – Eau Claire charted the workflow necessary for adding a new database for the McIntyre Library to best utilize BP Logix.

NASIG 2012

Case Study V

Carol Tenopir (2010) looks at ROI of the value of e-journals to grants income.

• “I could not submit as many grants. With grant funding at 4-6% of submitted proposals I would not have achieved my current funding level.” (U.S. University)

• “I guess on average the online access saves me more than 10 hours per week.” (Western European Research Institute)

(p. 44).

NASIG 2012

Case Study VI-?

What You or Your Colleagues are Doing

(Audience Provides At Least One Example.)

NASIG 2012

References

Bob Schufreider & Sion Romaine (2008): Making Sense of your Usage Statistics, The Serials Librarian, 54:3-4, 223-227, from http://dx.doi.org/10.1080/03615260801974164

Bryan Vogh & Hans Kishel (2011): E-forms: Making Workflows Work, UMWUG Conference in Fargo, ND, October 17, 2011, from http://people.uwec.edu/voghbs/UMWUG_E-forms_Handout.pdf

Carol Tenopir (2010): Measuring the Value of the Academic Library: Return on Investment and Other Value Measures, Serials Librarian, 58:1-4, 39-48, from http://dx.doi.org/10.1080/03615261003623005

Linda Hulbert, Dani Roach & Gail Julian (recorder)(2011): Integrating Usage Statistics into Collection Development Decisions, The Serials Librarian, 60:1-4, 158-163, from http://dx.doi.org/10.1080/0361526X.2011.556027

Mary Ann Trail, Kerry Chang-FitzGibbon & Susan Wishnetsky (recorder)(2012): Using Assessment to Make Difficult Choices in Cutting Periodicals, The Serials Librarian, 62:1-4, 159-163, from http://dx.doi.org/10.1080/0361526X.2012.652931

Nancy Beals & Marcella Lesher (2010): Managing Electronic Resource Statistics, The Serials Librarian, 58:1-4, 219-223, from http://dx.doi.org/10.1080/03615261003625844

Sarah Glasser & Michael A. Arthur (recorder) (2011): When Jobs Disappear: The Staffing Implications of the Elimination of Print Serials Management Tasks, The Serials Librarian, 60:1-4, 109-113, fromhttp://dx.doi.org/10.1080/0361526X.2011.556447

Thomas E. Nisonger (2008): The “80/20 Rule” and Core Journals, The Serials Librarian, 55:1-2,62-84, from http://dx.doi.org/10.1080/03615260801970774

NASIG 2012

Thank You!

Stephanie H. WicalAssistant ProfessorPeriodicals and Electronic Resources LibrarianMcIntyre LibraryUniversity of Wisconsin – Eau Claire105 Garfield AvenueEau Claire, WI 54702(715) [email protected]

Hans F. KishelAssistant ProfessorResearch and Instruction Librarian for ScienceMcIntyre LibraryUniversity of Wisconsin – Eau Claire105 Garfield AvenueEau Claire, WI 54702(715) [email protected]