Top Banner
Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session 34
54

Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Dec 24, 2015

Download

Documents

Kory Stone
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Crunching Numbers: OPAC Log Analysis of WebVoyage

Bennett Claire PonsfordDigital Services Librarian

Texas A&M University Libraries

EndUser 2007 Session 34

Page 2: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Overview

Why analyze your log files? How to do it What we found The changes we made What do the latest logs say? Improvements needed

Page 3: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Why Analyze? To see how your users search when

you’re not watching To resolve internal disagreements

over default searches, limits, etc. To see whether changes to

WebVoyage really improved search results

As a counterpoint to task-based user testing

Page 4: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

C.S. Lewis Lewis, C. S. (Clive Staples) LION WITCH? LION, WITCH? LION, WITCH, AND WARDROBE? Lewis, C. S. (Clive Staples)

Page 5: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.
Page 6: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Issues to Think About

Does Voyager capture the data you need? Privacy concerns

Does your network organize data the way you need? Staff vs. public IP addresses

Do you want all searches or a sample?

Page 7: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

How To

Read the documentation Technical Manual, Chapter 15, Popacjob

Begin logging your data Extract data into Access database

Clean up data as needed Run queries Scratch head and contact Tech

Support

Page 8: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.
Page 9: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

5-MAR-07 WebOpac 20061016102711 Title keyword(TKEY New) AND (TKEY York) AND(TKEY Times) Y West Campus Library K YN 7 1 W999.999.99.999AMDB20020820112825 N

Page 10: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Data Fields

Search_date

5-MAR-07

Stat_string WebOpacSession_id 20070305123912Search_type Title keywordSearch_string

(TKEY New) AND (TKEY York) AND (TKEY Times)

Page 11: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Data Fields (cont.)

Limit_flag YLimit_string

West Campus Library

Index_type KRelevance YHyperlink NHits 7

Page 12: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Data Fields (cont.)

Search_tab 1Client_type WClient_ip XXX.XXX.XX.XXXDbkey AMDB20020820112825Redirect Flag

N

Page 13: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.
Page 14: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

SQL for Count of Search TypeSELECT [Spring 2007 OPAC log].Search_type,

Count([Spring 2007 OPAC log].Search_type) AS CountOfSearch_type

FROM [Spring 2007 OPAC log]WHERE ((([Spring 2007 OPAC

log].Client_type)="W") AND (([Spring 2007 OPAC log].Search_tab)="1") AND (([Spring 2007 OPAC log].Hyperlink)=“N")AND (([Spring 2007 OPAC log].Client_ip) Not Like "128.194.8[4-7].*" And ([Spring 2007 OPAC log].Client_ip) Not Like "165.91.39.*"))

GROUP BY [Spring 2007 OPAC log].Search_typeORDER BY [Spring 2007 OPAC log].Search_type;

Page 15: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Results Author Browse – 334 Author headings – 943 Author keyword – 1587 Builder – 6 Call Number Browse – 777 Command – 16 Documents Call Number –

15 Expert keyword (rel) – 5 Expert keyword (rev) –

2378 Journal title keyword – 793 Journal title – 1375

Keyword – 82 Keyword (Relevance) --

6179 Keyword Search – 1574 LC Call Number browse --

14 Locally Assigned Call#-- 2 Simple Search -- 58 Subject Browse -- 851 Subject headings -- 128 Subject Hds keyword -- 251 Title keyword -- 2045 Title Redirect -- 615 Title starts with – 1677

Page 16: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

June 2006 (Voyager 5)

Page 17: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

September 2006 (Voyager 5)

Changed interface Defaults

Kept Tab at Simple Search Changed Search to Keyword (CMD*

with javascript) Changed result sort to by relevance

Page 18: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Fall 2006 Preparing to upgrade to Voyager

6.1 New keyword searches with ^ to

automatically AND words together Some people unhappy with recent

changes Default search Search results sort order

Decided to look at the data

Page 19: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Decisions upgrading to V6

Basic data Where are our searchers What search tab are they using How are they searching

Default search Order of title searches Simple limits

Page 20: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Where Are Our Searchers?

35%

65%

InsideLibraries

OutsideLibraries

Page 21: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

What Search Tab Used?

0%

20%

40%

60%

80%

100%

Simple (1) Keyword (akaBuilder - 2)

Course Reserves(3)

Inside Libraries Outside Libraries

Page 22: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Default Search: Discussion

Title search (TALL) What we traditionally had used Reference’s preference

General keyword search (new GKEY^*) What users are used to in a Google

world More forgiving search

Page 23: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Comparison of Searches Used

0

2000

4000

6000

8000

Inside Libraries Outside Libraries

Page 24: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.
Page 25: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Default Search: Decision

General keyword search (new GKEY^*) User preference Fewer No Hit results

Page 26: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

First Title Search: Discussion

Left anchored title (TALL) Preferred by Reference

Title keyword (new TKEY^*) More forgiving

Page 27: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Title Search (TALL): Problems

Page 28: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Title Search: Decision

Title keyword Left-anchored title had too many

problems

Page 29: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Simple Limits

Several additional location limits requested

Concern that too many would be confusing

Page 30: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Search Limits Used

0%

2%

4%

6%

8%

10%

Page 31: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Simple Limits: Decision

Added new limits and will evaluate with more data

Page 32: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.
Page 33: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Analysis of Voyager 6 Logs

Search frequencies No hit frequencies Title search problems Journal title search hits Search limits

Page 34: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Keyword and Subject Searches

0%

10%

20%

30%

Expertkeyword

Keyword Subjectbrowse

Subjectkeyword

Inside Libraries Outside Libraries

Page 35: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Author Searches

0.0%

2.0%

4.0%

6.0%

8.0%

AuthorBrowse

Authorheadingsbrowse

Authorkeyword

Inside Libraries Outside Libraries

Page 36: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Title Searches

0%

10%

20%

30%

Journalkeyword

Journaltitle

Titlekeyword

TitleRedirectKeyword

Title

Inside Libraries Outside Libraries

Page 37: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

No Hit Title Searches: Do We Own Them?

0%

10%

20%

30%

40%

50%

Yes No Unable toverify

Other

Keyword Left-Anchored (preliminary)

Page 38: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

No Hit Title Searches: Problems

0%

20%

40%

60%

Keyword Left-Anchored (preliminary)

Page 39: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Location Limits Used

0

100

200

300

400

500

600Media Services

Qatar

Cushing

Bestsellers

West Campus

Reference Coll.

Curriculum Coll.

Web Resources

Page 40: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Comparison of Limits

0%2%

4%6%

8%

Inside Libraries Outside Libraries

Page 41: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Have Changes Helped?

Search frequency No hits percentage

Page 42: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Search Frequency

0%

20%

40%

60%

Summer 2006 Fall 2006 Spring 2007

Page 43: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

No Hit Percentages Some

improvement but no major change

0%

5%

10%

15%

20%

25%

30%

Summer 2006 Fall 2006 Spring 2007

Page 44: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Detailed No Hits Percentages

0%

10%

20%

Summer 2006 Fall 2006 Spring 2007

Page 45: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Improvements: Spelling Spellchecking Automatic searching of variant

spellings “&” or “and” British vs. American spellings Numbers Abbreviations

Did you mean? Suggestions based on field

Page 46: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Improvements: Help

More granular no hits help Specific search types Any search with “conference” or

“proceedings” in it Journal title searches including “vol.”,

“no.”, or a number Searches with more than 4 or 5 words

More granular help for too many hits

Page 47: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Improvements: Specific Searches

Keyword searches Automatic stemming Ignore punctuation and spacing Ignore stop words

Title searches Ignore initial article

Page 48: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Journal search results layout

Whether to include the index field in the journal title search results Same search results but the order of

the results is change by the inclusion of the index field

Primarily a problem for single word titles that retrieve more than 1 screen of results – Science, Nature, etc.

Page 49: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.
Page 50: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.
Page 51: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

JALL Search ResultsResults

Count Percent Count Percent

No hits 252 32.5% 449 32.7%1 hit 144 18.6% 203 14.8%2-5 hits 264 34.0% 557 40.6%5-50 hits 76 9.8% 126 9.2%51+ hits 40 5.2% 37 2.7%

Total 776 1372

Inside Library Outside Library

Page 52: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

What Next?

Continued analysis of searches with no hits

Analysis of search repair strategies Word counts

Page 53: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

More Information Jansen, Bernard J. “Search log analysis:

What it is, what’s been done, how to do it,” Library & Information Science Research, 28 (2006) 407-432.

Yu, Holly and Margo Young, “The impact of Web search engines on subject searching in OPAC”, Information Technology and Libraries, 23 (2004) 168-180.

Page 54: Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session.

Contact Information

Bennett Claire Ponsford [email protected] 979/845-0877

https://libcat.tamu.edu