Beyond Page Level Metrics

B E Y O N D PA G E L E V E L M E T R I C S

Buddy Brewer @bbrewer

Philip Tellis @bluesmoon

G I T H U B

https://github.com/lognormal/beyond-page-metrics

https://github.com/lognormal/boomerang

git clone <clone url>

W H AT D O E S A PA G E L O O K L I K E O N T H E N E T W O R K ?

H O W D O D I F F E R E N T B R O W S E R S H A N D L E PA R A L L E L I Z AT I O N ?

W H I C H PA G E C O M P O N E N T S A F F E C T P E R C E I V E D L AT E N C Y ?

A R E A N Y O F T H E M S P O F S ?

• Static JavaScript files, external CSS files

• Anything that blocks onload if you have scripts that run on onload

W H Y D O W E N E E D R U M ?

C A N ’ T W E G E T T H I S A L R E A D Y ?

San Francisco London

Paris

Gilroy Tellisford

Eze

Fast Connections

Slow Connections

Common Browsers

Uncommon Browsers

≠

N AV I G AT I O N T I M I N GP E R F O R M A N C E T I M I N G

N AV I G AT I O N T I M I N G AVA I L A B I L I T Y

• IE >= 9

• FF >= 7

• Chrome >= 6

• Opera >= 15

• Latest Android, Blackberry, Opera Mobile, Chrome for Android, Firefox for Android, IE Mobile

N AV I G AT I O N T I M I N G E X A M P L E

var loadEventDuration = performance.timing.loadEventEnd - ! performance.timing.loadEventStart;

R E S O U R C E T I M I N GP E R F O R M A N C E T I M E L I N E

R E S O U R C E T I M I N G AVA I L A B I L I T Y

• IE >= 10

• Chrome

• Opera >= 16

• Latest Opera Mobile, Chrome for Android, IE Mobile

R E S O U R C E T I M I N G G E T S U S I N T E R E S T I N G T H I N G S

• Generate a complete waterfall https://github.com/andydavies/waterfall

• Calculate a cache-hit-ratio per resource

• Identify problem resources

C O R S : C R O S S - O R I G I N R E S O U R C E S H A R I N G

• Cross-domain resources only tell you start & end time

• Timing-Allow-Origin: *

L I M I TAT I O N S O F R E S O U R C E T I M I N G

• Does not report resources that error out, which is one of the things we care about

• Doesn’t tell you if a response is a 304 or 200

C AV E AT A B O U T T E S T I N G W I N D O W. P E R F O R M A N C E

• On Firefox 31, checking window.performance in an anonymous iframe throws an exception

• So we tried: if (“performance” in window) {}

C AV E AT A B O U T T E S T I N G W I N D O W. P E R F O R M A N C E

• But jslint complains about that

• So we switched to: if (window.hasOwnProperty(“performance")) { // I know right? }

C AV E AT A B O U T T E S T I N G W I N D O W. P E R F O R M A N C E

• Which does not work on Internet Explorer 10+!#

• So we ended up with: try {     if ("performance" in window && window.performance)        ... } catch(e) {    // WTF }

M E A S U R I N G X H R Sfunction instrumentXHR()!{!! var proxy_XMLHttpRequest,!! orig_XMLHttpRequest = window.XMLHttpRequest,!! readyStateMap;!!! if (!orig_XMLHttpRequest) {!! ! // Nothing to instrument!! ! return;!! }!!! readyStateMap = [ "uninitialized", "open", "responseStart", "domInteractive", "responseEnd" ];!!! // We could also inherit from window.XMLHttpRequest, but for this implementation,!! // we'll use composition!! proxy_XMLHttpRequest = function() {!! ! var req, perf = { timing: {}, resource: {} }, orig_open, orig_send;!!! ! req = new orig_XMLHttpRequest;!!! ! orig_open = req.open;!! ! orig_send = req.send;!!! ! req.open = function(method, url, async) {!! ! ! if (async) {!! ! ! ! req.addEventListener('readystatechange', function() {!! ! ! ! ! perf.timing[readyStateMap[req.readyState]] = new Date().getTime();!! ! ! ! }, false);!! ! ! }!!! ! ! req.addEventListener('load', function() {!! ! ! ! perf.timing["loadEventEnd"] = new Date().getTime();!! ! ! ! perf.resource.status = req.status;!! ! ! }, false);!! ! ! req.addEventListener('timeout', function() { perf.timing["timeout"] = new Date().getTime(); }, false);!! ! ! req.addEventListener('error', function() { perf.timing["error"] = new Date().getTime(); }, false);!! ! ! req.addEventListener('abort', function() { perf.timing["abort"] = new Date().getTime(); }, false);!!! ! ! perf.resource.name = url;!! ! ! perf.resource.method = method;!!! ! ! // call the original open method!! ! ! return orig_open.apply(req, arguments);!! ! };!!! ! req.send = function() {!! ! ! perf.timing["requestStart"] = new Date().getTime();!!! ! ! // call the original send method!! ! ! return orig_send.apply(req, arguments);!! ! };!!! ! req.performance = perf;!!! ! return req;!! };!!! window.XMLHttpRequest = proxy_XMLHttpRequest;!}

M E A S U R I N G X H R Sfunction instrumentXHR{!! var proxy_XMLHttpRequest! orig_XMLHttpRequest ! readyStateMap!! if (!orig_XMLHttpRequest! ! // Nothing to instrument! ! return! }!!! readyStateMap !! // We could also inherit from window.XMLHttpRequest, but for this implementation,! // we'll use composition! proxy_XMLHttpRequest ! ! var!! ! req !! ! orig_open ! ! orig_send !! ! req! ! !! ! ! ! req! ! ! ! ! perf! ! ! !! ! !!! ! ! req! ! ! ! perf! ! ! ! perf! ! !! ! ! req! ! ! req! ! ! req!! ! ! perf! ! ! perf!! ! !! ! !! ! };!! ! req! ! ! perf!! ! !! ! !! ! };!! ! req!! ! return! };!!! window.XMLHttpRequest }

In Short: Proxy XMLHttpRequest Capture open(),send() and events

M E A S U R I N G A S I N G L E O B J E C T

var url = 'http://www.buddybrewer.com/images/buddy.png';!var me = performance.getEntriesByName(url)[0];!var timings = { ! loadTime: me.duration, ! dns: me.domainLookupEnd - me.domainLookupStart, ! tcp: me.connectEnd - me.connectStart, ! waiting: me.responseStart - me.requestStart, ! fetch: me.responseEnd - me.responseStart!}

M E A S U R I N G A C O L L E C T I O N O F O B J E C T S

var i, first, last, entries = performance.getEntries();!for (i=0; i<entries.length; i++) {! if (entries[i].name.indexOf('platform.twitter.com') != -1) {! if (first === undefined) ! first = entries[i];! if (last === undefined) ! last = entries[i];! if (entries[i].startTime < first.startTime) ! first = entries[i];! if (entries[i].responseEnd > last.responseEnd) ! last = entries[i];! }!}!console.log('Took ' + (last.responseEnd - first.startTime) + ' ms');

T I M E B Y I N I T I AT O R T Y P E

function timeByInitiatorType() {! var type, res = performance.getEntriesByType("resource"), o = {};! for (var i=0;i<res.length;i++) {! if (o[res[i].initiatorType]) {! o[res[i].initiatorType].duration += res[i].duration;! if (res[i].duration > o[res[i].initiatorType].max) o[res[i].initiatorType].max = res[i].duration;! if (res[i].duration < o[res[i].initiatorType].min) o[res[i].initiatorType].min = res[i].duration;! o[res[i].initiatorType].resources += 1;! o[res[i].initiatorType].avg = o[res[i].initiatorType].duration / o[res[i].initiatorType].resources;! } else {! o[res[i].initiatorType] = {"duration": res[i].duration, "resources": 1, "avg": res[i].duration, "max": res[i].duration, "min": res[i].duration};! }! }! return o;!}

F I N D T H E S L O W E S T R E S O U R C E S O N T H E PA G E

function findSlowResources(ms, num) {! var res = performance.getEntriesByType("resource"), arr = [], i;! for (i=0; i<res.length; i++) {! if (res[i].duration > ms) arr.push(res[i]);! }! arr.sort(function(a,b){ return b.duration - a.duration });! return arr.slice(0, num);!}

F I N D P O T E N T I A L S P O F S

function findPossibleSpofs(ms) {! var res = performance.getEntriesByType("resource"), spofs = [];! for (var i=0;i<res.length;i++) {! var isSpof = true;! for (var j=0;j<res.length;j++) {! if (res[i].name != res[j].name && ! (res[j].startTime > res[i].startTime && res[j].startTime < res[i].responseEnd) ||! (res[j].endTime > res[i].startTime && res[j].endTime < res[i].responseEnd) ||! (res[j].startTime < res[i].startTime && res[j].endTime > res[i].responseEnd)) {! isSpof = false;! }! }! if (isSpof && res[i].duration > ms) spofs.push(res[i]);! }! return spofs;!}

This code is just an example, however it has O(n2) complexity, which might be very slow running in production.

F I N D S L O W H O S T S

function findPerfByHost() {! var res = performance.getEntriesByType("resource"), obj={};! for (var i=0;i<res.length;i++) {! var start = res[i].name.indexOf("://")+3,! host = res[i].name.substring(start),! end = host.indexOf("/");! host = host.substring(0,end);! if (obj[host]) {! obj[host].resources += 1;! obj[host].duration += res[i].duration;! if (res[i].duration < obj[host].min) obj[host].min = res[i].duration;! if (res[i].duration > obj[host].max) obj[host].max = res[i].duration;! obj[host].avg = obj[host].duration / obj[host].resources;! }! else {! obj[host] = {"duration": res[i].duration, "min": res[i].duration, "max": res[i].duration, "avg": res[i].duration, "resources": 1};! }! }! return obj;!}

U S E R T I M I N GP E R F O R M A N C E T I M I N G

U S E R T I M I N G AVA I L A B I L I T Y

• IE >= 10

• Chrome >= 25

• Opera >= 15

• Latest Opera Mobile, Chrome for Android, IE Mobile

U S E R T I M I N G E X A M P L E

performance.mark(‘event_start');!!setTimeout(function() {! performance.mark('event_end');! performance.measure(‘time_to_event’);! performance.measure('event_duration','event_start',‘event_end');! console.log('Event took ' + ! performance.getEntriesByName(‘event_duration')[0].duration + ! ' ms');!}, 1000);

P E R F O R M A N C E M A N A G E M E N T I N T H R E E S T E P S

How Fast Am I? How Fast Should I Be? How Do I Get There?

H O W FA S T S H O U L D I B E ?

T R A C K I N G C O N V E R S I O N SW H A T I S A C O N V E R S I O N ?

Orders Shares, Likes, Comments

Page Views Subscriptions

Signups Card Additions

Video Plays

M E A S U R I N G T H E I M PA C T O F S P E E DS P E E D S T R O N G LY C O R R E L A T E S T O C O N V E R S I O N S

T H I S M E A N S W E C A N M E A S U R E PAT I E N C E

E X A M P L E

Time Range: 1 Month

Median Load Time: 4.12

Visits: 25M

Conversion Rate: 2.71%

Average Order: $100

C A N W E D O B E T T E R ?S P E E D I N C R E A S E S D R I V E B U S I N E S S I M P R O V E M E N T S

Median Load Time: 4.12 Total Conversion Rate: 2.71% Conversion Rate @ 3.0s: 4.88%

W H AT A R E W E P L AY I N G F O R ?

Total Conversion Rate: 2.71%

Best Case Conversion Rate: 4.88%

Conversion Gap: 2.32%

Visits: 25M

AOV: $100

(4.88% - 2.71%) * 25M * $100 = $54.25M

1 second = $54M

BUT

1 0 0 T H P E R C E N T I L E ?P O T E N T I A L V S R E A L I S T I C G O A L S

Median Load Time: 4.12 Total Conversion Rate: 2.71% Conversion Rate @ 3.0s: 4.88%

R E A L I S T I C , I T E R AT I V E G O A L S

Target Load Time: 4 seconds (vs 3 seconds)

Percentile at 4 sec: 49th

Target Percentile: 60th (vs 100th percentile)

Percentile Gap: 11%

(4.88% - 2.71%) * (11% * 25M) * $100 = $6M

Improving from 4.12 sec @ 50th percentile

to 4.0 sec @ 60th percentile

= $6M / month

Thank You

AT T R I B U T I O N S

https://secure.flickr.com/photos/torkildr/3462607995 (servers) https://secure.flickr.com/photos/hackny/8038587477 (real users) https://secure.flickr.com/photos/isherwoodchris/3096255994 (NYC) https://secure.flickr.com/photos/motoxgirl/11972577704 (Countryside) https://secure.flickr.com/photos/98640399@N08/9287370881 (Fiber Optic) https://secure.flickr.com/photos/secretlondon/2592690167 (Acoustic Coupler) https://secure.flickr.com/photos/jenny-pics/2904201123 (Rum Bottle) https://secure.flickr.com/photos/bekathwia/2415018504 (Privacy Sweater) https://secure.flickr.com/photos/zigzaglens/3566054676 (Star Field)