Metrics and Analytics, Guest Lecture, UCLA

Analytics & MetricsUCLA Extension (Guest Lecture)

Darren Levy- Senior Director of Revenue, Retention at Mylife.com- Founder of Gatherspace.com

[email protected]

True or False:

Technology is the hardest part of building

an online business?

True or False:

Knowing the metrics is someone else’s job.

Breakdown of today’s discussion

- Part 1 – Definitions, why they are so critical- Part 2 – What metrics to focus on?- Part 3 – Optimizing metrics- Part 4 – Case Studies

I. Why do we need metrics?

What are metrics, why do we need them?

- Small units of measurement, typically ratio’s.- How are we doing? - Is business healthy?- Is the customer happy, is the product working?

99.1% Pure

…and what about analytics?

- Combining metrics, finding patterns of behavior.- What should we be working on? - Should we pivot?

Google Analytics – be cautious

- Storing all of your proprietary data with Google- Limitations on customizations- When shit hits the fan, getting customer support- You NEED to slice and dice and query your data

Why are metrics so importantfor YOU?!

- Be the Expert, be the Hero, the “go to” person- Don’t rely so much on others- Because we’re all Liars

Ice Cream Glove

Tightly ingrained withinLean Startup Process

Hypothesis

Experiment

Metrics

Validation

II. What metrics to focus on?

Vanity Metrics

- Not actionable- Not understandable- Examples:

• traffic • time on site• cumulative registrations• # of visits• Overall conversion rate

Single Key MetricOMTM

- Analysis Paralysis (don’t overdo it!)- For startups, business rally around a metric- Changes over time as business progresses

Customer Lifecycle – 5 Steps to SuccessAARRR!

Dave McClure’s Pirate MetricsAARRR!

AARRR Lifecycle

Acquisition Visit Site(or landing page, or widget)

* CPC* Keyword Rank* Ad CTR

CTR ~100%Value=.01

Category Lifecycle Experience Metrics Est Value

Activation Jump through initial hoop, signup (“a ha” moment)

* CTR CTR ~5%Value=$3

Retention Repeat Visitor, Open up emails

* Avg # of Visits* Engagement Score

CTR ~2%Value=$5

Revenue Customer generates revenue* RPV* AOV* ARPU

CTR ~1%Value=$25

Lean AnalyticsGrowth Stages (“Gates”)

Single Key MetricsFor Startups

How much do we spend to get a customer?Cost per Acquisition ($) CPA

How much do our customers pay us?Average Revenue per User ($) ARPU

At what rate are customers cancelling?Churn (%) Churn

How much is a customer worth over time?Lifetime Value per Customer ($)

LTV

Subscription v. Transactional Models

Subscription Business Transactional Business

VPR (value per reg)Churn / LTV / ARPUActivation & Engagement

Shopping Cart AbandonmentAverage Order Value (AOV)Upsell & Product Page CTR

III. How do we do this?

How do we get Metrics?

- Google Analytics- DIY – recording events, activities in a SQL db- AB testing- Other tools – Heatmaps

Types of AB Testing

- Simple AB testing- Multivariate testing- Price testing

Simple AB Testing

- Changing 1 element at a time?- Effecting CTR at any point in a funnel

Multivariate Testing

- More simultaneous variables- Takes longer to test- Better results than “simple”

Price Testing

- Testing different prices or terms- Not measuring CTR, measuring RPV- Pay attention to “Accumulation of Revenue”- Pricing Strategies (freemium, subsc., bundling)

Ugly Friend Concept

Famous Price Test (Ugly Friend)

- Decoy (ugly friend) pushes price selection- In this example, decoy pushed sales by 30%

Apple iPhone Pricing

~67% ~20% ~13%

10 Things you MUST Test

1. Calls to Action (CTA)2. Propositions3. Copy & Content4. Visual Media5. Funnel testing6. Forms 7. Emails 8. Pricing9. Shipping 10. Personalization

AB Test Example

Cohort AnalysisMeasuring engagement and value over time

Optimization via Heatmaps

Quantitative v. Qualitative Metrics

- Example of qualitative (fire alarm in middle of night, 99.97% up time, yet pissed off customers)- Why you need both. (user testing, surveys, call your customers)

IV. Case Studies

Price Test @ Dollarshaveclub.com

Control Test

4 7

7% higher CLTV4.4% higher 24 hr Net

Case Study: Netflix & House of Cards

- Goal to be “HBO” of online TV- Cost of $100m on production

- Analytics told them * Most customers watched David Fincher’s social network all the way* British version of “House of Cards” was highly watched* Significant overlap of British House of Cards watch Kevin Spacy films and Fincher films.

- Data so far * Q1 2013, brought in 3m new custmers* in Q1 alone, paid for cost of production* Increasing CLTV

Q&A