Amanda Spink : Analysis of Web Searching and Retrieval

Amanda Spink : Analysis of Web

Searching and Retrieval

Larry ReeveINFO861 - Topics in Information Science

Dr. McCain - Winter 2004

2

Background

Amanda Spink Self-described areas of work:

Information Retrieval Web Retrieval Human Information Behavior / Information

Seeking Medical Informatics

Ph.D. 1993 – Rutgers University Thesis - Feedback in Information Retrieval Studied under Tefko Saracevic

3

Background

Amanda Spink Over 140 papers published

5th in journal article production, 18th in citation production among U.S. IS faculty

Institute for Information Science – most highly cited paper in Web Retrieval: Real Life, Real Users, Real needs: A Study and Analysis of

User Queries on the Web (2000)

4

Background

Amanda Spink Associate Professor at University of Pittsburgh

School of Information Sciences

Prior faculty positions Pennsylvania State University

School of Information Science & Technology Web Research Group

University of North Texas School of Library and Information Sciences

5

Background

Tefko Saracevic

Associate Dean School of Communication, Information and Library

Studies, Rutgers University

Related research Test and Evaluation of IR systems Relevance in Information Science Analysis of web queries

6

Web Searching and Retrieval

Analyze user queries Important for building future IR systems on Web

Focus on search terms Failure analysis in query construction Term Relevance Feedback (TRF) Topics / Classification Use of language

7

Studies Conducted

U.S. – Excite (www.excite.com) “51K study”

51,473 queries 18,113 users March 9, 1997

“1M study” 1,025,910 queries 211,063 users September 16, 1997

8

Studies Conducted

European - AllTheWeb.com 1 million queries 200,000 users

Logs from two days: February 6, 2001 May 28, 2002

Most users from Norway and Germany

9

Studies Conducted

Issues with Web transaction logs Where does session start and end?

Temporal boundary – Spink found 15 mins avg, Others found 5mins, 12mins, 32mins, and 2 hours

Numerical boundary – 100 entries

How to eliminate non-individual users Meta-search engines, other agents

No user insight into user’s process

10

Findings

Relevance Feedback Advanced Search Techniques Term Characteristics Query Classification American vs. European

11

Findings: Relevance Feedback Term Relevance Feedback (TRF) rarely used

51K study 1,597 queries from 823 users (<5% of queries)

Those using TRF had longer sessions Successful 60% of time Implications:

Failure rate of 40% may be too high IR designers could automatically perform TRF

12

Findings: Relevance Feedback Mediated searching

11% of search terms come from TRF 37% from users, 63% from mediators 2/3 of TRF contributed positively

13

Findings: Relevance Feedback Identified 6 session states

Initial Query, Modified Query, Next Page, New Query, Relevance Feedback, Prev Query

Identified 4 session patterns Using the 6 session states

Implication: IR designers should accommodate these states and patterns

14

Findings: Relevance Feedback

Relevance Feedback Session Patterns

15

Findings: Advanced Search Techniques Includes:

Boolean operators Modifiers +, - Quotes (phrases)

Not often used by Web users, but used more by mediated search Boolean <10%, Modifiers 9%, 6% phrases

Used incorrectly Boolean: AND:50%, OR:28%, AND NOT:19% Modifiers: 75% of time Phrases: 8%

Users and advanced techniques do not get along!

16

Findings: Advanced Search Techniques Boolean, most common problems:

Not capitalizing AND Confusing ‘AND’ operator with ‘and’ conjunction

e.g. Science and Technology Science AND Technology

Modifiers, most common problems: Prefix rather than mathematical postix

+news +weather rather than news+weather No space required, as is required with Boolean

17

Findings: Term Characteristics Terms per query

1: 26.6%, 2: 31.5%, 3: 18.2%, >7: 1.8% Mediated searching: 7-15 terms

Distribution of terms not quite Zipf: Top terms account for 10% of all terms Single-use terms account for 9% of all

terms Not understood why this occurs

18

Findings: Query ClassificationClassification of queries based on Rutgers’ Web Classification

19

Findings: Query Classification What users are looking for is not what is on

Web: Distribution of content:

83% Commercial, 6% Educational, 3% Health Example: 10% of searches are for Health

Searchers find classifications understandable IR system presentation design

20

Findings: American & European Searching Commonalities:

Three or fewer terms American: 80%, European 85%

Predominantly use English terms Relevance judgments: less than 15 minutes

viewing retrieved documents Information seeking sessions short

21

Findings: American & European Searching Differences

Categories American: Entertainment, Sex, Commerce European: People-places-things, Computers, Commerce

American searchers spent more time searching e-commerce sites than European counterparts

Did not examine: Use of advanced techniques Relevance feedback First in initial set of studies?

22

Findings: Summary

Number of query terms is about 2 TRF is not used often Boolean operators and modifiers not used

often – difficulty in using them correctly Users do not spend much time making

relevancy judgments Term frequency distribution is a few terms

used often, many terms used only once

23

Findings: Summary

Most users had single query only and did not follow up with successive queries

Average viewing of 2 pages 50% did not access beyond first page; more

than 75% did not go beyond 2 pages

24

Implications / Further Research Improve use of advanced search techniques

UI changes, Venn Diagrams Improve use of relevance feedback

Automatic generation of TRF results Improve classification of results

UI changes, result overview Improve understanding of language use

Adapt IR designs to language Examine cultural differences

TRF, advanced search techniques (same or different)

25

Amanda Spink -Web Searching and

Retrieval Questions