Top Banner

Click here to load reader

User Engagement: from Sites to a Network of Sites or The Network Effect Matters!

May 10, 2015



  • 1.User Engagement:from Sites to a Network of Sitesor The Network Effect Matters! Ricardo Baeza-YatesMounia LalmasYahoo! Labs Barcelona Joint work with Janette Lehmann and Elad Yom-Tov and many others at Yahoo! Labs -1-

2. Outlineo Motivation, definition and scopeo Models of user engagemento Networked user engagement-2- 3. Motivation, Definition and Scopeo Definition and scopeo Characteristics of user engagemento Measures of user engagemento Our research agenda-3- 4. User Engagement connecting three sidesQuality of the user experience that emphasizes the positive aspects of the interaction, and in particular the phenomena associated with users wanting to use a web application longer and frequently.Successful technologies are not just used, they areengaged withuser feelings: happy, sad,user mental states: concentrated, user interactions: click, readexcited, bored, challenged, lost, interested comment, recommend, buy, The emotional, cognitive and/or behavioural connection that exists, at any point in time and over time, between a user and a technological resource S. Attfield, G. Kazai, M. Lalmas and B. Piwowarski. Towards a science of user engagement (Position Paper), WSDM Workshop on User Modelling for Web Applications, 2011.-4- 5. Would a user engage with this web site? -5- 6. Would a user engage with this web site? (art event calendar) -6- 7. Would a user engage with this web site? (e-commerce skating)-7- 8. Would a user engage with this web site? (music repository) -8- 9. Would a user engage with this web site? (photographer)-9- 10. Characteristics of user engagement (I) Users must be focused to be engagedFocused attention Distortions in the subjective perception of time used to measure it Emotions experienced by user are intrinsically motivating Positive Affect Initial affective hook can induce a desire for exploration, active discovery or participation Sensory, visual appeal of interface stimulates, promote Aestheticsfocused attention Linked to design principles (e.g. symmetry, balance, saliency) People remember enjoyable, useful, engaging experiences and want to repeat themEndurability Reflected in e.g. the propensity of users to recommend an experience/a site/a product - 10 - 11. Characteristics of user engagement (II) Novelty, surprise, unfamiliarity and unexpectedNovelty Appeal to user curiosity, encourages inquisitive behavior and promotes repeated engagement Richness captures the growth potential of an activity Richness and control Control captures the extent to which a person is able to achieve this growth potential Trust is a necessary condition for user engagement Reputation, trust and Implicit contract among people and entities which is expectation more than technological Motivation, interests, incentives, and Difficulties in setting up laboratory style experiments benefits - 11 - 12. Forrester Research The four Is Presence of a userInvolvement Measured by e.g. number of visitors, time spent Action of a userInteraction Measured by e.g. CTR, online transaction, uploaded photos or videos Affection or aversion of a user Intimacy Measured by e.g. satisfaction rating, sentiment analysis in blogs, comments, surveys, questionnaires Likelihood a user advocates Influence Measured by e.g. forwarded content, invitation to joinMeasuring Engagement, Forrester Research, June 2008- - 12 13. Peterson et al Engagement measure - 8 indicesClick Depth Index: page viewsDuration Index: time spentRecency Index: rate at which users return over timeLoyalty Index: level of long-term interaction the user haswith the site or product (frequency)Brand Index: apparent awareness of the user of the brand,site, or product (search terms)Feedback Index: qualitative information includingpropensity to solicit additional information or supply directfeedbackInteraction Index: user interaction with site or product(click, upload, transaction)Peterson etal. Measuring the immeasurable: visitor engagement, WebAnalyticsDemystified, September 2008 - 13 - 14. Measuring user engagementMeasures Characteristics Self-reportedQuestionnaire, interview, report, Subjective, engagement product reaction cards user study (lab/online)Mostly qualitative CognitiveTask-based methods (time spent, Objective, engagement follow-on task) user study (lab/online)Neurological measures (e.g. EEG)Mostly quantitativePhysiological measures (e.g. eyeScalability an issue?tracking, mouse-tracking) InteractionWeb analytics + data scienceObjective, engagement (CTR, bounce rate, dwell time, etc) data studyMetrics and user models QuantitativeLarge scale- 14 - 15. Interaction engagement Online metricsProxy of user engagement- 15 - 16. Diagnostic and what we can doDiagnostic: work exists, but fragmented.In particular: o What and how to measure depend on services and goals o Going beyond site engagement What we have done: 1. Models of user engagement 2. Networked user engagement 3. Complex networks analysis Future: Economic model for networked UE - 16 - 17. Models of User Engagement Online sites differ concerning their engagement! Games Search Users spend Users come much time per frequently and visit do not stay long Social mediaSpecial Users comeUsers come on frequently andaverage once stay long Service News Users visit site, Users come when needed periodicallyis it possible to model these differences andcompare different classes of sites? - 17 - 18. Data and MetricsInteraction data, 2M users, July 2011, 80 USA sites Popularity #Users Number of distinct users#VisitsNumber of visits#ClicksNumber of clicks Activity ClickDepth Average number of page views per visit.DwellTimeA Average time per visit LoyaltyActiveDays Number of days a user visited the siteReturnRate Number of times a user visited the siteDwellTimeL Average time a user spend on the site. - 18 - 19. Diversity in User EngagementEngagement of a site depends on users and timeUsers and Loyalty Time and PopularitySites have different user groupsSite engagement can be periodic orcontains peaksProportion of user groups is site- dependent mail, social media media (special events) media, entertainmentdaily activity, shopping,navigationentertainment- 19 - 20. MethodologyGeneral models User-based models Time-based modelsDimensions5 user groupsweekdays, weekend8 metrics 8 metrics per user 8 metrics per time spangroup#Dimensions8 4016Kernel k-means with Kendall tau rank correlation kernelNum. of clusters based on eigenvalue distribution of kernel matrix Significant metric values with Kruskal-Wallis/Bonferonni#Clusters(Models) 675Analyzing cluster centroids = models - 20 - 21. Models of user engagement (I) Models based on engagement metrics 6 general models Popularity, activity and loyalty are independent from each other Popularity and loyalty are influenced by external and internal factors e.g. frequency of publishing new information, events, personal interests Activity depends on the structure of the siteinterest-specificperiodicmediae-commerce,configuration- 21 - 22. Models of user engagement (II) Models based on engagement metrics, user and time User-based [7 models] Time-based [5 models] Models based on engagement per user Models based on engagement over group weekdays and weekendnavigation game, sport hobbies, daily news interest-specific Sites of the same type (e.g. mainstream media) do not necessarily belong to the same model The groups of models describe different aspects of engagement, i.e. they are independent from each other- 22 - 23. Relationships between modelsGroups of models are independent from each other General User TimeGeneral 0.003.50 4.23User3.500.00 4.25Variance of InformationTime4.234.25 0.00[0,5.61]Example:Model mu2 [high popularity and activity in all user groups, increasing loyalty] 50% to model mt2 [high popularity on weekends and high loyalty on weekdays] 50% to model mt3 [high activity and loyalty on weekends] - 23 - 24. Recap & NextUser engagement is complex and standard metrics capture only a part of itUser engagement depends on users and timeFirst step towards a taxonomy of models of userengagement and associated metricsNextInteraction between modelsUser demographics, time of the day, geo-location, etcJ. Lehmann, M. Lalmas, E. Yom-Tov and G. Dupret. Models of User Engagement, UMAP 2012. - 24 - 25. Understanding the problem:Users on Yahoo! network of sites - 25 - 26. Networked user engagement: engagement across a network of sites Large online providers (AOL, Google, Yahoo!, etc.) offer not one service (site), but a network of services (sites) Each service is usually optimized individually, with some effort to direct users between them Success of a service depends on itself, but also on how it is reached from other services (user traffic) Measuring user engagement across a network of sites should account for user traffic between sites. - 26 - 27. Online multi-tasking leaving a site is not a bad thing!users spend more and more of their online session multi-tasking, e.g. emailing,reading news, searching for information ONLINE MULTI-TASKINGnavigating between sites, using browser tabs, bookmarks, etcseamless integration of social networks platforms into many services- 27 - 28. Online multi-taskingUsers switch between sites within an online session (several sites are visited and the same site is visited several times)NavigationBackLink Other1 Browser usage changed button Less usage of back button1995 35.7% 45.7% 18.6%1997 31.7% 43.4% 24.9%2006 14.3% 43.5% 42.2% Oberdorf et al 1) Usage of tabs, bookmarks, typing the URL directly, 2) 28 - 29. Online multi-taskingUsers switch between sites within an online session (several sites are visited and the same site is visited several times)Navigation Browser usage changed More and more usage of tabsBackLinkOther1 b