Web-Based Usage Surveys MINES for Libraries TM Brinley Franklin University of Connecticut [email protected] Terry Plum Simmons GSLIS [email protected] CNI Fall 2005 Task Force Meeting Phoenix, AZ December 5, 2005
Jan 03, 2016
Web-Based Usage SurveysMINES for LibrariesTM
Brinley FranklinUniversity of Connecticut
Terry PlumSimmons GSLIS
CNI Fall 2005 Task Force Meeting Phoenix, AZ
December 5, 2005
2CNI Fall 2005 Task Force Meeting
Overview
• Research problem– Assessment of networked electronic services– Open access – beyond vendor supplied data– Issues with web-based surveys in assessing the use of networked
electronic services – Digital libraries – how to assess
• Project– MINES for LibrariesTM
• Web based usage and user survey– Assessment infrastructure and gateway
• Examples – data from:– Medical libraries– Large academic libraries– Consortia
3CNI Fall 2005 Task Force Meeting
Assessment of networked services• Vendor supplied data - standards
– COUNTER - Counting Online Usage of Networked Electronic Resources
• http://www.projectcounter.org/
– ICOLC – International Coalition of Library Consortia • http://www.library.yale.edu/consortia/
– ISO – International Standards Organization• ISO 11620 Library Performance Indicators • http://www.iso.org/
– NISO – National Information Standards Organization• NISO Z39.7 Library Statistics• http://www.niso.org/
– ARL E-Metrics• http://www.arl.org/stats/newmeas/emetrics/index.html
– are helpful and trusted• LibQUAL+TM
– Suite of services that libraries use to solicit, track, understand, and act upon users’ opinions of service quality.
• http://www.libqual.org/index.cfm
• DigiQUALTM
– Digital library service quality – set of 180 items tested against five collections
4CNI Fall 2005 Task Force Meeting
Open access --Beyond vendor supplied data
• ISO standards for the electronic collection include e-books, electronic databases, ejournals, and digital documents. – Free internet resources are counted separately. – Yet, library liaison and subject web pages include free resources.
• Other electronic resources are important, including:– Digital libraries – local, archival resources– Pre-print and post-print servers– Open access journals– Open access repositories (e.g., institutional repositories)– Electronic theses and dissertations servers
• Can we assess use of these resources as well as vendor supplied databases and ejournals?
5CNI Fall 2005 Task Force Meeting
Web-based usage surveys• Advantages
– Can go beyond gross usage counts, using sampling– User and use information
• E.g.; can query purpose of use, the discipline or affiliation, the status of the patron– Opinion
• Service quality perception– Can be tied to usage
• Point of use– Can fold in other services (e.g., ILL, Ask A Librarian)
• Methodological issues– Sample, not census– Replicable?– Truly random sample is difficult to obtain– High non-response rate – are the responses representative of the population?
• Survey not seen• Survey seen, but not filled out
– Users, not usage– Remembered, predicted or critical incident reports – perceived to be not as
reliable as vendor census based on usage– Browsers can influence the display of the web survey
• And more. See for example:• Holly Gunn, “Web-based Surveys: Changing the Survey Process”, First Monday, Issue
7, 2002. • D.A. Dillman, Mail and Internet Surveys, The Tailored Design Method. 2nd ed., 2000.
6CNI Fall 2005 Task Force Meeting
MINES for LibrariesTM
• MINES is a transaction-based research methodology consisting of a web-based survey form and a random moments sampling plan.
• MINES typically measures who is using electronic resources, where users are located at the time of use, and their purpose of use.
• MINES measures usage as well as users.
• MINES is administered at the point of use, and surveys repeat usage during the sample period.
• MINES was adopted by the Association of Research Libraries (ARL) as part of the “New Measures” toolkit in May, 2003.
• MINES is different from other electronic resource usage measures that quantify total usage (e.g., COUNTER, E-Metrics) or measure how well a library makes electronic resources accessible (LibQUAL+TM).
7CNI Fall 2005 Task Force Meeting
MINES for LibrariesTM
8CNI Fall 2005 Task Force Meeting
Assessment Infrastructure
On Campus Patrons
In Library Patrons
Off campus patrons
?
9CNI Fall 2005 Task Force Meeting
Assessment Gateway
online catalogfederated searching
10CNI Fall 2005 Task Force Meeting
Demographics by Location of UserU.S. Main Libraries
24%
25%
8%
43%
Graduate Students
Faculty, Staff,Research Fellows
UndergraduateStudents
All Other Users
40%
31%
25%
4%
28%
11%
21%
40%
On Campus, Not in the Libraryn = 8,838
Inside the Libraryn = 9,172
Off-Campusn = 7,688
MINES for Libraries™
11CNI Fall 2005 Task Force Meeting
31%
46%
6%
17%
Graduate Students
Faculty, Staff,Research Fellows
UndergraduateStudents
All Other Users
Off-Campusn = 5,133
Demographics by Location of UserU.S. Medical Libraries
Inside the Libraryn = 6,819
On Campus, Not in the Libraryn = 19,582
20%
53%
1%
26%
34%
32%
5%
29%
MINES for Libraries™
12CNI Fall 2005 Task Force Meeting
MINES for LibrariesTM
14%
14%
69%
3%
Graduate Students
Faculty, Staff,Research Fellows
UndergraduateStudents
All Other Users
Off-Campusn = 9,163
Demographics by Location of UserOntario Council of University Libraries
Inside the Libraryn = 4,047
On Campus, Not in the Libraryn = 7,090
45%
24%
29%
2%31%
16%
48%
5%
13CNI Fall 2005 Task Force Meeting
25%
27%9%
39%
SponsoredResearch
Instruction
Patient Care
Other
On-Campus, Not in the Libraryn = 15,423
In the Libraryn = 6,312
Purpose of Use By LocationMedical Libraries
2003 – 2005
Off-Campusn = 4,375
*83% of sponsored research usage occurred outside the library. 92% of this use took place on-campus.
Overall Usen = 26,110
31%
14%10%
45%
34%37%
13%16%
14%
49%
18%19%
14CNI Fall 2005 Task Force Meeting
Purpose of Use By Location Main Campus Libraries
2003 – 2005
5%
66%
3%
26%
Sponsored Research
Instruction
Other
Other SponsoredActivities
In the Libraryn = 7,618
On-Campus, not in the Libraryn = 6,641
11%63%
24%
2%
All Usagen = 19,271
•72% of sponsored research usage of electronic resources occurred outside the library; 83% of this took place on campus.
6%
59%
33%
2%
Off-Campusn = 5,012
21%63%
14%2%
15CNI Fall 2005 Task Force Meeting
MINES for LibrariesTM
64%
36%
Inside the Library
Outside the Library
U.S. Main LibrariesTotal Users
n = 25,698
21%
79%
U.S. Medical Libraries
Total Usersn = 31,883
80%
20%
OCUL (Canada)
LibrariesTotal Users
n = 20,300
Location of Electronic Resources Users
16CNI Fall 2005 Task Force Meeting
17CNI Fall 2005 Task Force Meeting
OCUL Data from MINES
18CNI Fall 2005 Task Force Meeting
Web-Based Usage SurveysMINES for LibrariesTM
[email protected]@simmons.edu
http://www.arl.org/stats/newmeas/mines.html