Using RUM to deliver optimal Performance in the Fast New MSN Paul Roy, Microsoft Buddy Brewer, SOASTA @bbrewer
Aug 18, 2015
Using RUM to deliver optimalPerformance in the Fast New MSN
Paul Roy, Microsoft
Buddy Brewer, SOASTA@bbrewer
New MSN•New UI design
• Entire stack on Azure
•New geo-distributed Content Store
• Responsive design
• Single web codebase for all verticals
Launched Fall 2014 in 55 countries
RUM Adoption
• Complement to synthetic measurements
•Adoption phases• Evaluation
• Pre-Launch
• Launch
• After Launch
• Futures
Played a key role in delivering optimal performance
Evaluation• Evaluated internal & external solutions
•MSN contacted SOASTA
• Proof of Concept working within 1 week
• Loved the dashboards & real-time
• Cost effective
• Secured buy-off from senior mgmt
Launch •War Room – all hands on deck throughout the
night
• Perf guys were there to watch traffic ramp-up
and verify performance
After Launch
•Using data on ongoing basis to watch trends
and generate regular scorecards against
targets
• Spot regressions
•Verify improvements
• Reporting to executives
Steady state
Types of Problems
• Code regressions not detected by controlled
synthetic environment with limited data points
• Country-specific problems
• Browser-specific problems
• Device-specific problems
• Problems at high percentiles
• Problems at low bandwidths
• RUM waterfalls exposed anomalous script
behavior in the wild
Detected by RUM but not synthetic
Futures• Reduce time to detect regressions (alerting
and more frequent tracking)
•Mining waterfall and resource-level data to
detect anomalous patterns
• Further study of correlation between speed
and user engagement
Where do we go next
Correlation with Session Length
Visits: 100MCurrent pages/visit: 1.5Median load time: 2 secPages/visit @ 1.5 sec: 2
(2 – 1.5) * 100M = 50M PV