Anagha Kulkarni Carnegie Mellon University Jaime Teevan ...translectures.videolectures.net/site/normal_dl/tag=... · Anagha Kulkarni Carnegie Mellon University Jaime Teevan, Krysta

Anagha KulkarniCarnegie Mellon University

Jaime Teevan, Krysta M. Svore, Susan DumaisMicrosoft Research

WSDM 2011Hong Kong, Feb. 9-12, 2011

Web search is strongly influenced by time

Queries’ popularity changes over time

Document content changes over time

Query intent (what’s relevant to the user) changes over time

Take away:

Changes in query popularity and

document content can signal

changes in query intent.

Changes in

Query

Popularity

Changes in

Document

Content

Changes in

Query

Intent

http://ncaa.com becomes relevant during the basketball tournament but not before or after

Before During After

http://mahalo.com/march-madness is always about March Madness and is relevant before, during, and after

Before, During, After

100 temporal queries, 20 documents per query

Type of change targeted

Δ query popularity Δ relevant results

Ye

ar

20

09

20

10

Queries that

spiked during this time

period last year

(april fools,

tax extension)

Queries with high

click entropy during this

time period last year

(miss usa, easter

baskets)

Queries related

to scheduled

upcoming events

(ipad, crystal bowersox)

Queries anticipated to

be likely to have

breaking news

(earthquake, lady gaga)

March 25 – May 28, 2010

Query

Frequency (daily)

Document

Number of clicks (per query, daily)

Result rank (per query, daily)

Crawl of document content (daily)

Query Intent

Human relevance judgments (weekly)

Changes in

Query

Popularity

Changes in

Document

Content

Changes in

Query

Intent

Methods

Measure query frequency over time

Characterize query popularity curves

Number of spikes / Periodicity / Shape / Trend

Changes in

Query

Popularity

Changes in

Document

Content

Changes in

Query

Intent

0

0,002

0,004

0,006

0,008

0,01

0,012

0,014

3-25 4-01 4-09 4-15 4-22 4-29 5-07 5-13 5-21 5-28

Qu

ery

Po

pu

lari

ty (

no

rmal

ize

d)

Date

Query Popularity - march madness

During tournament

After tournament

march madnessMultiple spikes / not periodic /

castle / downward trend

0

0,02

0,04

0,06

0,08

0,1

0,12

0,14

0,16

0,18

3-25 4-01 4-09 4-15 4-22 4-29 5-07 5-13 5-21 5-28

Qu

ery

Po

pu

lari

ty (

no

rmali

zed

)

Date

Query Popularity - dancing with the stars

dancing with the starsMultiple spikes / periodic / castle / flat

0

0,01

0,02

0,03

0,04

0,05

0,06

0,07

0,08

3-25 4-01 4-09 4-15 4-22 4-29 5-07 5-13 5-21 5-28

Qu

ery

Po

pu

lari

ty (

no

rmali

zed

)

Date

Query Popularity - mlb

Pre-season During season

mlbMultiple spikes / not periodic /

castle / upward trend

0

0,05

0,1

0,15

0,2

0,25

0,3

3-25 4-01 4-09 4-15 4-22 4-29 5-07 5-13 5-21 5-28

Qu

ery

Po

pu

lari

ty (

no

rmali

zed

)

Date

Query Popularity - miley cyrus

New music video

miley cyrus1 spike / not

periodic / wedge /

flat

0

0,01

0,02

0,03

0,04

0,05

0,06

0,07

0,08

0,09

3-25 4-01 4-09 4-15 4-22 4-29 5-07 5-13 5-21 5-28

Qu

ery

Po

pu

lari

ty (

no

rmali

zed

)

Date

Query Popularity - earthquake

earthquake1 spike / not periodic /

right sail/ flat

Baja CA and Mexico quake

0

0,002

0,004

0,006

0,008

0,01

0,012

0,014

3-25 4-01 4-09 4-15 4-22 4-29 5-07 5-13 5-21 5-28

Qu

ery

Po

pu

lari

ty (

no

rmali

zed

)

Date

Query Popularity - tax extension

tax extension1 spike / not periodic

/ left sail / flat

Tax day

Most queries in study

One or more spikes

Not periodic

Wedge

Show temporal trend

Changes in

Query

Popularity

Changes in

Document

Content

Changes in

Query

Intent

Changes in

Query

Popularity

Changes in

Document

Content

Changes in

Query

Intent

1

10

100

1000

3-25 4-01 4-09 4-15 4-22 4-29 5-07 5-13 5-21 5-28

log(

TF)

mar

ch m

adn

ess

on

pag

e

Date

Content Change - march madness

http://www.mahalo.com/march-madness

http://www.ncaa.com/

During tournament After tournament

61% pages show query-related (TF) change

95% pages changed by less than 15% (Dice)

Periodic and multiple-spike queries are more likely to have content-changed documents

Changes in

Query

Popularity

Changes in

Document

Content

Changes in

Query

Intent

Changes in

Query

Popularity

Changes in

Document

Content

Changes in

Query

Intent

0

1

2

3

4

5

3-25 4-01 4-09 4-15 4-22 4-29 5-07 5-13 5-21 5-28

Hu

man

Re

leva

nce

Ju

dgm

en

ts

Date

Human Judgments - march madness

During: scores/schedules After: general info/planning

Click entropy is correlated with

Spikes

1 spike has lower click entropy than Multiple spikes

Shape

Sail (taxes online, pgatour) has significantly lower click entropy than wedge (robert pattinson, kristen stewart)

Trend

Downwards has lower click entropy

than flat (microsoft update)

Changes in

Query

Popularity

Changes in

Document

Content

Changes in

Query

Intent

Changes in query popularity and document content are related to changes in query intent

• Spikes

• Shape

• Trend

• Click

entropy

• Periodicity

• Human

Relevance

Understanding query intent can be used to improve

Captions, Ranking, Results display

Patterns of query intent change

Zoom: march madness – intent changes from general to specific and back to general

Shift: opening day – different intents throughout year

Static: easter ideas – same intent throughout year

Questions?