LibMeter LibMeter Striving for Excellence with Electronic Library Statistics Indicators for Performance Evaluation Introduction – Current Practice Institut für Informationswissenschaft, University of Applied Sciences, Cologne (FH-Köln IIW) Seminar on Library Evaluation (S. FÜHLES-UBACH) Peter Ahrens Guest Lecture V 1.00a – 2009-05-18 Cologne, 7th May 2009
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
LibMeterLibMeter
Striving for Excellence with Electronic Library Statistics
Indicators for Performance EvaluationIntroduction – Current Practice
Institut für Informationswissenschaft, University of Applied Sciences, Cologne (FH-Köln IIW)
Seminar on Library Evaluation
(S. FÜHLES-UBACH)
Peter Ahrens Guest Lecture
V 1.00a – 2009-05-18
Cologne, 7th May 2009
LibMeterLibMeter
LibMeter Seminar
This LibMeter Introduction Seminar is a continuation from the following
Slideshare Presentation: Current Practice – Basics
It also relates to: LibMeter Case Study #1(University of Düsseldorf)
Roadmap of this PresentationIntroduction - Current Practice
I. Introduction - Basics A Quick Futuristic Web Tour Usage Stats for the Rest of Us (No Maths please!) Library Role, Services & Usage
II. Introduction – Current Practise Where to get Usage Figures from ? How to interpret usage Figures ? LibMeter Case Studies
III. Perspectives
LibMeterLibMeter
Where to get Usage Figures for
Electronic Services?
LibMeterLibMeter
German Library Statistics
DBS 2007+
LibMeterLibMeter
Which Parties are involved in Generating Service and Stats ?
Sorry, we need to become technical again – Stats Source One Party-Scenario (Examples: OPAC hits, Homepage)
Service Provider = Library += Raw Statistics Provider = Webserver/ IT Department
(+/- allied special statistics service provider) Two Party-Scenario (Example: COUNTER stats)
Lib. + Content Provider = Online Publisher/ Service Host Three Party Scenario (Examples EZB; OpenURL)
Lib. + Content + Web Service provider - Library Computing Service usually Server with powerful integrated specific statistic tools
Multi Party-Scenario: (Example BIX Virtual Library Use) Lib. + Content + Web Service + Stats Provider
LibMeterLibMeter
DBS Starts reporting on Electronic Usage in 2007
DBS Variable Analysis (VA) is available since 2004 DBS-VA contains annual data back to 1999 DBS reports on 6 electronic „OUTPUT“-parameters
(=use or usage) since 2007
LibMeterLibMeter
Use of Electronic Resources & Online Services in DBS (2007)
General Services (1 Party or 1 Party+) Visits of Library-Homepage (Zählpixel) Visits of OPAC (Zählpixel) *1.)
Content Delivery (2 Parties) *2.)
Full Text Downloads of locally licensed articles (COUNTER) Database Search Sessions (+/-COUNTER) Fulltext Views of locally managed Electronic Documents
A = free accessB = locally licensedC = As B but only certain yearsD = paid full text, not licensed
Access Category of Publication
LibMeterLibMeter
Pros and Cons of counting EZB „free“ eJournal-Linking activity ?
NO (B&C only) - Controlling & accounting approach (DBS 2007) „Free“ eSources do not comply with usage indicator DBS#181 definition - to
have clearly locally attributable cost & effort Direct Cost is a pre-requisit for DBS#181 = No direct cost, no count !
„Maßgeblich ist immer die Prämisse, dass diese Angebote der Bibliothek Aufwand (Geld und Arbeit) verursachen und damit eine mit den vorhandenen Ressourcen erbrachte Leistung darstellen“ (*)
Content 1-to-1 correlated with (paid!) COUNTER articles DBS#183
YES (+A&D): Service monitoring approach (EZB basic Report) Library participates in community effort of coop. cataloguing to keep EZB alive Service is clearly attributable to local users of a given institution Many additional Service events for similar service (+50%)
Reporting would allow analysis of trends in usage of electronic scholarly information and significant related library services (abd community efforts) What is the bulk use of all NatLis, all Open Access journals ? Are there differences between Institution(s) (Types) Are there Proportion Shifts between A, B, C & D over time (Berlin-Declaration, DFG)
LibMeterLibMeter
Looking annually at 1 Parameter (EZB)
at 1 time (DBS 2007) for many libraries
LibMeterLibMeter
Comparing EZB Use (B+C) among University Libraries in 2007
Compiled from DBS VA: http://www.bibliotheksstatistik.de/eingabe/dynrep/index.php [2009-04-15]
EZB (B+C only) call as per DBS 2007Standardized per Primary User per Year
2007 EZB Usage per Primary User at Universities versus Polytechs
EZB Statistics Universities (n=57)
Median = 6,2 Mean / Average = 6.9 Standard Deviation = 5.6 Variation Coefficient = 0.8
Polytechs (n=34) Median = 0,42 (sic!) Mean / Average = 2.0 Standard Deviation = 6.1 Variation Coefficient =3.0
Düsseldorf: Mean / Average 7.0 = 0.13 Std. Dev. above Mean
While you may calculate Mean and St. Dev. for a population, make sure it is normally
distributed before drawing further conclusions (check Median) !
LibMeterLibMeter
Looking at Distribution among University Libraries & Polytechs ?
A: Number of EZB calls correlated with conventional Loans* ? NO, there is no correlation
B: Number of EZB calls correlated with the Staff/Student ratio* ? YES, there seems to be a weak
positive correlation i.e. higher staff proportion goes
along with higher the EZB usage In other words (Hypothesis):
Staff uses EZB eJournal-Catalogue more than Students
This is not astonishing, but for the first time it becomes quantifiable
A
B
LibMeterLibMeter
BIX-2008+Performance Indicator
„Virtual Use“
LibMeterLibMeter
HDM Usage Counts for Electronic Library „Traffic“
Data: Courtesy of I. SIEBERT & J. KREISCHE, University Library Düsseldorf from private Website at HDM [2009-04-30]
LibMeterLibMeter
BIX Performance-Indicator „Virtual Library Use“ New (per 2008*) in German Library Performance Index „BIX“ Normalized Combined Parameter: „Virtual Annual Use“ Definition: (OPAC & Homepage) / Number of Primary
Users Simple Interpretation
Year Source OPAC Homepage OPAC + HP Primary Users2007 DBS 684.097 3.477.400 4.161.497 16.991
HDM 1.116.511 2.360.890 3.477.401 16.991
Hits per User 65,7 138,9 204,7 = Virtual Use
LibMeterLibMeter
Electronic Usage enters the „Main Stage“ in 2008 (with BIX)
From Electronic Library Output to „Virtual Use“ (DBS2007 > BIX2008)
Principle Take reliable Usage figures from
different relevant Services with high number of Events per User per Year
Combine to new parameter Benefits
Comprehensive & Easy to understand
Easy to compare E.g. with physical use
Limits Difficult to standardize
Homepage & OPAC counts across institutions (different software, no established counting standard)
HDM-Zählpixel-Provider limits
LibMeterLibMeterHow SMART is „Virtual Use“ ?Specific: Quite general Measurable: YES Community Standard with help
of HDM ZählpixelAchievable and Attributable: Library is actively producing Service Today every library can produce #Relevant and Realistic: YES Users frequently go there Closely tied to core services How representative for total spectrum
of electronic Library Services ?Time-bound, Timely, Trackable, and
Targeted: No log-files publicly available (yet?) Not available for all periods
LibMeterLibMeter
DBS & BIX StatisticsBenefits & Limits
Benefits Broad Institutions Base Fairly Comprehensive Creates Awareness
Community Public
Online Data Warehouse Transparency Allows Meta Studies
Limits Few Parameters Limited Scope
i.e. more detailled services missing
Standardization Problems Incomplete data Only Annual Data Time Lag International Comparability ?
LibMeterLibMeter
Journal & Database Vendors
(COUNTER)
LibMeterLibMeter
COUNTER provides „standardized“ Usage Stats
LibMeterLibMeter
Example: Usage of Subscribed JournalsVia COUNTER & SUSHI & ScholarlyStats
LibMeterLibMeter
COUNTER StatisticsBenefits & LimitsBenefits Working towards Standardization
of Online Content Usage Counting Accepted international
(commercial Providers) Code of Conduct
Available for hundreds of publishers & Database providers Available for most e-Journals
Basis for Commercial Discussions with Publishers and Hosts External: With providers Internal: With faculty
Limits Commercial Content only
OpenAccess, PubMed, Google Scholar missing
Expertise & work needed to handle the hundreds and thousands of spreadsheets
Needs Tools to handle Data XML-Harvester (SUSHI) Warehouse: ScholarlyStats
Standardization Problems No ISO Standard !
Focus on Publishers Content, not Library e-Services
Linking with OpenURL ResolverPrinciple & URL-Syntax
LibMeterLibMeter
Commercial Introduction of LinkResolvers (2000-2004)
Time since Market Introduction (in Years per Sept. 2004)(Source: Computers in Libraries, Okt. 2004)
0 1 2 3 4 5
Ex Libris - SFX
Elsevier/Endeavor - LinkFinderPlus
Openly - 1Cate
Innovative Interfaces - WebBridge
Geac - Vlink
Fretwell-Downing - OL2
Sirsi - Sirsi Resolver
Serials Solutions - Article Linker
EBSCO - LinkSource
Ovid - LinkSolver
TDNet - TOUR
LibMeterLibMeter
OpenURL Dynamically Links to the „Appropriate Copies“
LibMeterLibMeter
OpenURL Statistics available ad hocwith Breakdowns - SFX Admin Center
•Ermittlung der Link-Nutzung in der Datenbank CAPlus für Feb.-Apr. 06•zusammengefaßt monatlich•Eingeschränkt nur auf die Gruppe der (echten) universitären Nutzer•Ausgabe differenziert nach Dokumentenzahl zu denen elektronischer Volltext geboten werden konnte (incl. Nationallizenzen) versus kein elektronischer Volltext (nur andere SFX-Dienste der Bibliothek) •Anmerkung: Dieser Statistik-Typ erlaubt das einfache Abschätzen und Monitoring von Volltext-Abdeckungsgraden für verschiedene Datenbanken, Fachbereiche oder Anwenderungruppen.
LibMeterLibMeter
OpenURL StatisticsBenefits & LimitsBenefits Global ISO Standard Counts fully comparable Very wide scope of service Built-In Report Generators
possible Sources, Targets, Services … Variable Periods Sub-Groups of Users (IP-based or
other) Service and Context focussed (not
just content)
Limits Not all libraries have OpenURL
resolver yet (about 95 % of University Libraries do)
Not official DBS-parameter (yet?) Extra Cost of OpenRUL Resolver
Service Not all Links go through OpenURL
(same applies for other link types) Some sources do not send
source-ID -> remain unidentified
LibMeterLibMeter
Working more with Usage Figures
Case Studies
LibMeterLibMeter
LibMeter Case_Study #1
Data: Personal Communication May 2009, Courtesy of Dr. Kreische, ULBD Düsseldorf
Basics of Electronic UseBIX „Virtual Use“
@ ULBD Uni DüsseldorfSee separate Slide show
LibMeterLibMeter
Selected Readings to broaden and to deepen your Understanding
1. BIX - Der Bibliotheksindex 2008. B.I.T.online Sonderheft. 2008 Juni ;1-56. 2. Blecic DD, Fiscella JB, Wiberley Jr SE. Measurement of Use of Electronic Resources: Advances in Use
Statistics and Innovations in Resource Functionality. College and Research Libraries. 2007 ;68(1):26-44. 3. Ceynowa VK, Coners A. Balanced Scorecard für wissensch. Bibliotheken. Vittorio Klostermann; 2002. 4. Gallagher J, Bauer K, Dollar DM. Evidence-based librarianship: Utilizing data from all available sources to
make judicious print cancellation decisions. Lib. Collections, Acq. & Tech. Serv.. 2005 ;29(2):169-179. 5. George Boston, Whang M. E-Resources Usage Data: Apples to Oranges and Fixing Holes [Internet].
1853/20874/13/Boston_Whang_ERL_final.ppt+scholarly-stats+sfx&cd=4&hl=de&ct=clnk&gl=de&client=firefox-a 6. Hutzler E. Bibliotheken gestalten Zukunft : Kooperative Wege zur digitalen Bibliothek. Dr. Friedrich
Geißelmann zum 65. Geburtstag. Universitätsverlag Göttingen; 2008. 7. Kreische J. Die Messung von Vernetzung. Nutzungsstatistiken mit SFX [Internet]. Münster: 2006.
1. Available from: http://docserv.uni-duesseldorf.de/servlets/DocumentServlet?id=9728 7.8. Kreische J. Zwischen Ranking und Qualitätsmanagement: BIX WB [Internet]. Mannheim: 2008.
1. Available from: http://www.opus-bayern.de/bib-info/volltexte/2008/583/pdf/BIX%20Mannheim%20(Word%202003).pdf9. Poll R, te Boekhorst P, Measuring quality, Saur, München, 2007. 10. Lossau N. digital services in academic libraries. In: Digital Convergence - Libraries of the Future: Libraries
of the Future. Springer; 2008. 11. Shepherd P. The feasibility of developing and implementing journal usage factors: a research project
sponsored by UKSG. Serials. 2007;20(2):117-123.
LibMeterLibMeter
Special Thanks for critical & constructive discussions to: Joachim KREISCHE, ULB Düsseldorf Simone FÜHLES-UBACH, FH Köln Eric MULDER, Den Haag Marion MÜLLER , ULB Düsseldorf Jessica BUSCHMANN, Dortmund Nol VERHAGEN, UvA, Amsterdam Sylvia THIELE , UB Dortmund Britta RIEBSCHLÄGER , ULB Düsseldorf Peter KOSTÄDT, UB Köln
LibMeterLibMeter
About me – Key Fields of InterestsLibrariesEarly 1970s – Founding highschool libraryEarly 1980s - Converting Card Catalog to electronic (Mainframe & punchcards)Late 1980s creating bibliographic database for 5.000+ article reprints (Mac) Since 1990 consulting scholarly libraries on retrieval systems, End user friendynessHospital Libraries
Life Sciences& Statistics
Ph.D. thesis on pattern recognition in brain Several peer-reviewed publicatios
Presentations
Publishing & SoftwareEditing & Managing highschool journal for 5 yearsHeading Cycling Campaign Newsletter for 3 years
Knowledge Finder – Biomedical Search Engine •Distribution & Localization for Europe
12 years e- Ressources•Elsevier / Kluver / Ex Libris
•Science Direct,Embase.com, Ovid, SFX, MetaLib
Knowledge Management1990s Scientific Knowledge Management Software • Development & DistributionLate 1990s e-library transformation• consulting, Projects, trainingsMid 2000s Library e-services•Development of Cooperative strategies•Market Analysis & Business Planning
Information Technology in the SciencesLate 1970s first own programming on handheld computers, then Mainframes, then Apple II, then MacLate 1980s German Academic Software Price for co-developping a relational bibliographicpProgram - PARiS1990s Fuzzy logic, natural language & relevance ranked output retrievalLate 1990s – Online e-Journal and integrated search platformsMid 2000s – OpenURL, Context sensitive linking, Metasearch, ERM, Recommenders, Library 2.0
LibMeterLibMeter
LibMeter Seminar
x
This LibMeter Introduction Seminar is a continuation from the following
Slideshare Presentation: Current Practice – Basics
It also relates to: LibMeter Case Study #1(University of Düsseldorf)
LibMeterLibMeter
LibMeter Seminar
x
Personal Announcement:It is intended to continue this series slowly with further modules amongst others on:
Detecting, following and predicting Usage-Trends Measuring Impact of e-Service Marketing Events
International Library Comparisons Comparing Academic & Library Excellence