"Rum First" NYC Webperf Meetup

RUM First

Who I Am

Cliff Crocker

VP Product

SOASTA, Inc.

@cliffcrocker

http://www.flickr.com/photos/luc/

Real Users

http://www.flickr.com/photos/spullara/

Happy Users

http://www.flickr.com/photos/sybrenstuvel/

Not SoHappy

RUM 101Real User Measurement (RUM) is a technology for collecting performance metrics directly from the browser of an end user.

• Involves instrumentation of the page with JavaScript

• Measurements are fired across the network to a collection point through a small request object (beacon)

<JS>MySite.com

<beacon>

This helped…

C’mon Safari, be cool…

http://caniuse.com/nav-timing

69.34% Browser Support

Why RUM is Awesome

• Reduces the level of convincing we have to do within our organization that ‘faster is better’

• The only way to measure perceived performance from your end user

• Easy to maintain

• Only getting better…– Check out: http://blog.patrickmeenan.com/2013/07/measuring-

performance-of-user-experience.html– User Timing for RUM and Synthetic

What gives??

No RUM

RUM • According to a study done in 2012* only 1 in 5 of the top ecommerce sites are using some form of RUM

*http://www.webperformancetoday.com/2012/11/13/real-user-monitoring-rum-ecommerce-sites/

Challenges with RUM

1. It’s a “data” problem

2. People don’t like change

3. Hard to “sell” internally

For a modest sized site, in a given day

1-3 Million Page Views

Perf Timing

User Agent

OS/Device

Geography

Page Segmentation

Bandwidth Classification

A/B Bucket

‘Other’

Many have advocated sampling…

1% sample, looking at a period of about 3 hours for a given page segment (Checkout)

Filtered to the United States…

Using a sampling approach

Looking only at California…

…and IE 9 traffic only

Now look at a 100% sample for the same time duration

All the same filters applied – AND looking specifically at ‘Back-End’ time

Sampled at 1%

Not sampled

Many use a batch solution…

• Batch processing large amounts of data has been a traditional approach for RUM

– Great for looking at historical data, trending, etc.

– Big Data is not necessarily Fast Data

• Not ‘Operational’

In order for RUM to be operationally relevant, you need the data in real-time

Synthetic vs. RUM

http://www.flickr.com/photos/myvector/

It is Not Either Or

• Synthetic and RUM coexist in any performance strategy

• Great use cases for both

Synthetic Monitoring

• Pros

– Technology is great

– Real Browsers (IE, Chrome, FF)

– Built in Alerting

– Waterfall analysis

– Screen shots & headers

– Object level detail

• Cons

– False Positives

– Impossible/impractical to measure everything

– Fixed number of browser/OS combinations

– Simulated bandwidth constraints at best

– Too few data points for statistical relevance

RUM

Cast a wide net• Identify key areas of concern• Understand real user impact• Tie this back to $$ (next)

Synthetic

Diagnostic tool• Identify issues in a ‘lab’• Reproduce a problem found with

RUM

http://www.flickr.com/photos/84338444@N00/with/3780079044/

http://www.flickr.com/photos/ezioman/

RUM in the Workplace

Studies show…

However, you have to look at your own data

Understand your KPIs and How to Track Them

• Retail: Order Conversion, Add to Cart Conversion, Revenue

• Media: In Line Display Ad Views, Comments

• Other: Facebook likes/Social Media sharing

Bottom Line, if you want your business to care – then make it relevant for

them (speak their language)

Strategy for Success – RUM First

RUM First

• All the data

• Real-time and historical

Supplement with

Synthetic

• Reduce measurements based on RUM

• Use for adhocdiagnostic

Map to business

KPIs

• YOUR goals

• YOUR data

Free and Open Toolsets

RUM JS Collection Visualization Synthetic

boomerang.js Stats D Graphite WebPagtest

Episodes ShowSlow SpeedCurve (beta is free?)

SOASTA mPulse, Torbit, Google Analytics, Pingdom, New Relic

mPulse Litehttp://www.soasta.com/free/