Online Multitasking and User Engagement

ONLINE MULTITASKING AND USER ENGAGEMENT

CIKM 2013

In  collabora*on  with:  Mounia  Lalmas,    

Ricardo  Baeza-­‐Yates,    George  Dupret  

Jane%e  Lehmann  

outline

1.   Mo%va%on  How  do  users  browse  the  web  today?      

2.   Characteris%cs  of  online  mul%tasking  Ac2vity  during  and  between  visits      

3.   Measuring  online  mul%tasking  Defini2on  of  new  metrics,  case  study  

Lights  on  by  JC*+A!  

How  do  users  browse  the  Web  today?  

leC  by    [  embr  ]    

ONLINE MULTITASKING

4  JaneGe  Lehmann   Mo2va2on  

Browsing  the  “old  way”  

facebook   news   news  news   news   mail  

1min   2min   1min   3min  

Dwell  2me  during  a  visit  on  a  news  site:  7min  on  average  

news  site  

ONLINE MULTITASKING

5  JaneGe  Lehmann   Mo2va2on  

Nowadays  

news   facebook   mail  news   news   news  

1min   2min   1min   3min  

Dwell  2me  during  a  visit  on  a  news  site:  2.33min  on  average  (1min  |  3min  |  3min)  

ONLINE MULTITASKING

6  JaneGe  Lehmann   Mo2va2on  

•  Users  switch  between  sites,  to  do  related  or  totally  unrelated  tasks        •  E.  Herder  [1]:  

»  75%  of  sites  are  visited  more  than  once  »  74%  of  revisits  are  performed  within  a  session  

 Measuring  browsing  behavior  can  lead  to  incorrect  conclusions.    

[1]  E.  Herder.  Characteriza*ons  of  user  web  revisit  behavior.  In  LWA,  2005.  

Characteris%cs  of  online  mul%tasking  

Danboard's  Messy  Hom

e  by  Mullenkedheim

DATA SET

Interac%on  data  •  July  2012  •  2.5M  users  •  785M  page  views  

•  We  defined  a  new  naviga2on  model                                            (see  paper  for  detail)  

•  Categoriza2on  of  the  most  frequent  accessed  sites  (e.g.  mail,  news,  shopping)  »  11  categories  (news),  33  subcategories  (e.g.  news  

finance,  news  society)  »  760  sites  from  70  countries/regions  

8  JaneGe  Lehmann   Characteris2cs  

Visit activity

Visit  frequency    

9  JaneGe  Lehmann   Characteris2cs  

Mul%tasking  depends  on  the  site  under  considera%on    •  Social  media  sites  are  revisited  the  

most  

•  News  (tech)  sites  are  the  least    revisited  sites  

news (finance)

news (tech)

social media

mail

2.09

1.76

2.28

2.09

4.65

1.59

4.78

4.61

#Visits (avg sd)

Visit activity

Ac%vity  between  visits      

10  JaneGe  Lehmann   Characteris2cs  

Differences  in  the  absence  %me    •  50%  of  sites  are  revisited  aCer  less  

than  1min            -­‐  Interrup*on  of  a  task  

•  There  are  revisits  aCer  a  long  break              -­‐  Returning  to  a  site  to  perform  a  new  task  

0.00

0.25

0.50

0.75

1.00

10ï2 10ï1 100 101 102

mailsocial media

news (finance)news (tech)

Cum

ula

tive

pro

bab

ility

Absence time [min]

*   v2  v1   *   v3  *  -­‐  absence  2me  

Visit activity

Ac%vity  paLern      

11  JaneGe  Lehmann   Characteris2cs  

•  Four  types  of  "aGen2on  shiCs”  

•  Complex  cases  refer  to  no  specific  paGern  or  repeated  paGern  

•  Successive  visits  can  belong  together  (i.e.,  to  the  same  task)  

0.23

0.28

0.33

mail sites

news (finance) sites news (tech) sites

social media sites

decreasing attention increasing attention

constant attention complex attention

Pro

por

tion

of to

tal

dw

ell tim

e on

site

p-value = 0.09m = -0.01

p-value = 0.07m = -0.02

p-value = 0.79m = 0.00

0.23

0.28

0.33

Pro

por

tion

of to

tal

dw

ell tim

e on

site

Danboard  by  sⓘndy°  

Measuring    online  mul%tasking    

Cumulative activity

Cumula%ve  ac%vity          

                 vi  Browsing  ac2vity  during  the  ith  visit                    ivi  Browsing  ac2vity  between  the  (i-­‐1)th  and  ith  visit                    k=3  Rescaling  factor  for  ivi                    m  Browsing  ac2vity  (e.g.  dwell  2me,  page  views)  

 Assump%on:  If  users  return  aCer  short  2me,  they  return  to  con2nue  with  same  task.  If  users  return  aCer  longer  2me,  they  return  to  perform  a  new  task  -­‐  an  indica2on  of  loyalty  to  the  site.    

13  JaneGe  Lehmann   Metrics  

CumActm,k = log10 (v1 + ivik •vi

i=2

n

∑ )

iv2   v2  v1   iv3   v3  

Cumulative activity

Cumula%ve  ac%vity          

                 vi  Browsing  ac2vity  during  the  ith  visit                    ivi  Browsing  ac2vity  between  the  (i-­‐1)th  and  ith  visit                    k=3  Rescaling  factor  for  ivi                    m  Browsing  ac2vity  (e.g.  dwell  2me,  page  views)  

 Assump%on:  If  users  return  aCer  short  2me,  they  return  to  con2nue  with  same  task.  If  users  return  aCer  longer  2me,  they  return  to  perform  a  new  task  -­‐  an  indica2on  of  loyalty  to  the  site.    

14  JaneGe  Lehmann   Metrics  

CumActm,k = log10 (v1 + ivik •vi

i=2

n

∑ )

iv2   v2  v1   iv3   v3  

Cumulative activity

Cumula%ve  ac%vity          

                 vi  Browsing  ac2vity  during  the  ith  visit                    ivi  Browsing  ac2vity  between  the  (i-­‐1)th  and  ith  visit                    k=3  Rescaling  factor  for  ivi                    m  Browsing  ac2vity  (e.g.  dwell  2me,  page  views)  

 Assump%on:  If  users  return  aCer  short  2me,  they  return  to  con2nue  with  same  task.  If  users  return  aCer  longer  2me,  they  return  to  perform  a  new  task  -­‐  an  indica2on  of  loyalty  to  the  site.    

15  JaneGe  Lehmann   Metrics  

CumActm,k = log10 (v1 + ivik •vi

i=2

n

∑ )

iv2   v2  v1   iv3   v3  

v1  +  v2  +  v3      

Cumulative activity

Cumula%ve  ac%vity          

                 vi  Browsing  ac2vity  during  the  ith  visit                    ivi  Browsing  ac2vity  between  the  (i-­‐1)th  and  ith  visit                    k=3  Rescaling  factor  for  ivi                    m  Browsing  ac2vity  (e.g.  dwell  2me,  page  views)  

 Assump%on:  If  users  return  aCer  short  2me,  they  return  to  con2nue  with  same  task.  If  users  return  aCer  longer  2me,  they  return  to  perform  a  new  task  -­‐  an  indica2on  of  loyalty  to  the  site.    

16  JaneGe  Lehmann   Metrics  

CumActm,k = log10 (v1 + ivik •vi

i=2

n

∑ )

iv2   v2  v1   iv3   v3  

Cumulative activity

Cumula%ve  ac%vity          

                 vi  Browsing  ac2vity  during  the  ith  visit                    ivi  Browsing  ac2vity  between  the  (i-­‐1)th  and  ith  visit                    k=3  Rescaling  factor  for  ivi                    m  Browsing  ac2vity  (e.g.  dwell  2me,  page  views)  

 Assump%on:  If  users  return  aCer  short  2me,  they  return  to  con2nue  with  same  task.  If  users  return  aCer  longer  2me,  they  return  to  perform  a  new  task  -­‐  an  indica2on  of  loyalty  to  the  site.    

17  JaneGe  Lehmann   Metrics  

CumActm,k = log10 (v1 + ivik •vi

i=2

n

∑ )

iv2   v2  v1   iv3   v3  

v1  +  (iv2)3�  v2  +  (iv3)3�  v3      

Activity pattern

ALen%on  shiN  and  range          

                 n=4    Number  of  visits  in  session                      σ  Variance  in  the  visit  ac2vity                    μ  Average  of  the  visit  ac2vity                    inv  Modifica2on  of  the  “Inversion  number”    

   Descrip%on:  AGShiC  models  the  shiC  of  aGen2on  in  the  browsing  ac2vity  AGRange  describes  fluctua2ons  in  the  browsing  ac2vity  

18  JaneGe  Lehmann   Metrics  

AttShiftm,n =invm,n −min Invm,n

| max Invm,n |− | min Invm,n |AttRangem,n =

σ (Vm,n )µ(Vm,n )

Activity pattern

ALen%on  shiN  and  range  

19  JaneGe  Lehmann   Metrics  

-­‐1   0   1  

0  

constant   constant   constant  

>  0  

decreasing   complex   increasing  

AUen*on  shiV  

AUen*o

n  rang

e  

Comparing  the  ranking  of  the  sites  •  Visitdt  –  Dwell  2me  during  a  visit  •  Sessiondt  –  Dwell  2me  during  a  session                    Ø  Visitdt  and  Sessiondt  correlate  Ø  Otherwise  no  correla2on  à  the  other  metrics  capture  different  aspects  of  

browsing  behavior  

Comparing metrics

20  JaneGe  Lehmann   Metrics  

Visitdt   Sessiondt   CumActdt   ALShiNdt  

Sessiondt   0.57  

CumActdt   -­‐0.04   0.24  

ALShiNdt   0.09   0.22   0.02  

ALRangedt   -­‐0.01   -­‐0.01   -­‐0.26   0.19  

“Models”  of  browsing  behavior  

•  Clustering  of  sites  using  mul2tasking  and  standard  engagement  metrics:  •  CumActdt,  AGShiCdt,  AGRangedt  •  Visitdt,  Sessiondt    

•  We  iden2fied  five  cluster:                

Models of browsing behavior

21  JaneGe  Lehmann   Metrics  

C4: 74 sites

0.25

-0.25

0.75

-0.75

C5: 166 sites

0.25

-0.25

0.75

-0.75

C3: 156 sites

0.25

-0.25

0.75

-0.75

C2: 108 sites

0.25

-0.25

0.75

-0.75

C1: 172 sites

0.25

-0.25

0.75

-0.75

Visitdt

[min] CumActdt,3

AttShiftdt,4

AttRangedt,4

Sessiondt

[min]

Models of browsing behavior

22  JaneGe  Lehmann   Metrics  

Visitdt

[min] CumActdt,3

AttShiftdt,4

AttRangedt,4

Sessiondt

[min]

C2: 108 sitesauctions, front page,

shopping, dating

0.25

-0.25

0.75

-0.75

C1: 172 sitesmail, maps, news,

news (soc.)

0.25

-0.25

0.75

-0.75

One  task  during  a  session    §  High  dwell  2me  per  visit  and  during  

the  whole  session    §  Users  return  to  con2nue  a  task  (short  

absence  2me)    §  C1:  aGen2on  is  shiCing  to  another  site  §  C2:  aGen2on  is  shiCing  slowly  towards  

the  site  

C4: 74 sitesfront page, search,

download

C3: 156 sitesauctions, search,

front page, shopping

0.25

-0.25

0.75

-0.75

0.25

-0.25

0.75

-0.75

Models of browsing behavior

23  JaneGe  Lehmann   Metrics  

Several  tasks  during  a  session    §  Users  perform  several  tasks  on  these  

sites  during  a  session  

§  No  simple  ac2vity  paGern    

§  C3:  Dwell  2me  per  visit  is  low,  but  the  dwell  2me  per  session  is  high  

Visitdt

[min] CumActdt,3

AttShiftdt,4

AttRangedt,4

Sessiondt

[min]

C5: 166 sitesservice, download,

blogging, news (soc.)

0.25

-0.25

0.75

-0.75

Models of browsing behavior

24  JaneGe  Lehmann   Metrics  

Sites  with  low  ac%vity    §  Users  do  not  spend  a  lot  of  2me  on  

these  sites    §  Time  between  visits  is  short    §  AGen2on  is  shiCing  towards  the  site  

Visitdt

[min] CumActdt,3

AttShiftdt,4

AttRangedt,4

Sessiondt

[min]

C2: 108 sitesauctions, front page,

shopping, dating

0.25

-0.25

0.75

-0.75

C3: 156 sitesauctions, search,

front page, shopping

0.25

-0.25

0.75

-0.75

Models of browsing behavior

25  JaneGe  Lehmann   Metrics  

Browsing  behavior  can  differ  between  sites  of  the  same  category    §  C2:  users  visit  site  once  to  perform  

their  task  

§  C3:  users  visit  site  several  2mes  to  perform  task(s)  

Visitdt

[min] CumActdt,3

AttShiftdt,4

AttRangedt,4

Sessiondt

[min]

SUMMARY and Future Work

JaneGe  Lehmann   26  

•  Online  mul2tasking  affects  the  way  users  access  sites  –  Standard  metrics  do  not  capture  this!!!  

•  We  defined  metrics  that  describe  different  aspects  of  mul2tasking  •  CumAct  accounts  for  the  2me  between  visits  •  AGShiC,  AGRange  describe  aGen2on  shiCs  

•  We  showed  that  mul2tasking  depends  on  the  site  under  considera2on    

Future  work:  •  Can  we  improve  the  defini2on  of  a  task?  •  How  does  mul2tasking  affect  other  metrics,  such  as  bounce  rate  and  click-­‐

through  rate?  •  Does  mul2tasking  differ  in  different  countries?  

Summary  

Janette Lehmann Universitat Pompeu Fabra, Spain [email protected] Mounia Lalmas Yahoo Labs London [email protected] George Dupret Yahoo Labs Sunnyvale [email protected] Ricardo Baeza-Yates Yahoo Labs Barcelona [email protected]

Online Multitasking

+ User

Engagement