How to Be a Web 2.0 Metrics Jedi - Web 2.0 Expo (Apr 09)

8/14/2019 How to Be a Web 2.0 Metrics Jedi - Web 2.0 Expo (Apr 09)

1/90

Become a Web 2.0 Jedi:How to Use the Force of Product &

Marketing Metrics to Optimize YourBusiness

Web 2.0 ExpoApr 3, 2009

#w2e#metrics


2/90

Startup Metrics for Pirates:AARRR!

Web 2.0 Expo SFApril 2009

Dave McClure, Founders Fundhttp://www.foundersfund.com/

http://500hats.typepad.com/

http://slideshare.net/dmc500hats/
http://www.foundersfund.com/http://slideshare.net/dmc500hats/http://slideshare.net/dmc500hats/http://www.foundersfund.com/


3/90

Startup Metrics: The Basics

Measure Stuff. Keep It Simple.

5 Steps: Startup Metrics for Pirates (AARRR!)

3 Metrics Frameworks + 3 Roles (CEO, Dev/Product, Marketing)

Iterate & Optimize. Feedback Loop. (Loop. Loop.) One Step at a Time. The Ass vs. Face Issue.

Appendix(we wont have time)

Activation

Retention

Acquisition

Referral

Revenue


4/90

Web 2.0: HellYes, Good Times.

1. # Users, Bandwidth = Bigger.

2. Lower Startup Costs = Badder.

3. PPC, E-Com $ Growing = Uncut.

Collect Usage Metrics in Real-Time

Decisions Based on Measured User Behavior


5/90

The Startup Metrics Religion

Progress Features (Less = More)

Focus on User Experience

Measure Conversion; Compare 2+ Options Fast, Frequent Iteration (+ Feedback

Loop)

Keep it Simple & Actionable


6/90

Optimize 4 Happiness(both User + Business)

Define States of User + Business Value

Prioritize (Estimate) Relative Value of Each State

Move Users: Lower Value -> Higher Value Optimize for User Happiness / Business $$$

Achieve Low Cost + High Value @ Scale

$$$


7/90

Discover MeaningKeywords, Images, Call-to-Action

Top 10 - 100 words

Your Brand / Products

Customer Needs / Benefits

Competitors Brand / Products

Semantic Equivalents

Misspellings

Relevant images People

Products

Problems

Solutions

Call-to-Action

Words

Images

Context Button/Link

Emotion

Result

Positive? Negative?

Neutral (= Death)

A/B test & Iterate


8/90

Startup Metrics for Pirates: AARRR!

Acquisition:users come to site from various channels

Activation: users enjoy 1st visit: "happy experience

Retention: users come back, visit site multiple times Referral: users like product enough to refer others

Revenue: users conduct some monetization behavior

AARRR!

(note: If youre in a hurry,

watch 5 min video after slide

23)


9/90

AARRR!: 5-Step Startup Metrics Model

Website.com

Revenue$$

$

Biz DevAds, Lead Gen,Subscriptions,ECommerce

ACQUISITION

SEOSEM

Apps &Widgets

Affiliates

Email

PR BizDev

Campaigns,Contests

Direct,Tel, TV

SocialNetworks

Blogs

Domains

Retent

ion

Emails &Alerts

System Events &Time-based

Features

Blogs, RSS,News Feeds


10/90

Q: Whats My Business Model?

Can be one of the following:

1. Get Users (= Acquisition, Referral)

2. Drive Usage (= Activation, Retention)3. Make Money (= Revenue*)

* ideallyprofitable revenue

Note: eventuallyneed to turn Users/Usage -> Money


11/90

Just Gimme the GOOD Metrics.Users, Pages, Clicks, Emails, $$$...?

Q: Which of these is best? How do you know?

1,000,000 one-time, unregistered unique visitors

500,000 visitors who view 2+ pages / stay 10+ sec 200,000 visitors who clicked on a link or button

20,000 registered users w/ email address

2,000 passionate fans who refer 5+ users / mo.

1,000 monthly subscribers @ $5/mo

thegoodstuff.


12/90

3 Core Models:biz model, conversion dashboard, mktg channels

Define 1-Page Biz Model: customer segments + desired actions / behaviors

Identify critical Conversion Events & Dashboard for each segment & prioritize

Test & develop Marketing Channels; measure Volume (#), Cost ($), Conv (%)

Optimizeproduct & marketing using Fast Iteration Cycles & A/B Testing


13/90

Role: Founder/CEO

Q: Which Metrics? Why?

A: Focus on Critical Few Actionable Metrics(if you dont use the metric to make a decision, its not actionable)

Hypothesize Customer Lifecycle

Target ~3-5 Conversion Events (tip: Less = More)

Test, Measure, Iterate to Improve


14/90

The 1-Page Business Model(Users + Conversions + Priorities)

Q1: What types of people use your website? Visitor= Average User / Buyer

Contributor= Content Contributor / Seller

Distributor= Passionate Fan (unpaid) / Affiliate (paid)

Q2: What actions could they take to help you orthem ?


15/90

TeachStreet Metrics v1:Eye Chart Madness


16/90

TeachStreet 1-Page Business Model:Teachers & Students

Teachers Students

Activation

Claim ProfileAdd Class

Contact TeacherView 3 Pages

Retention

Visit 1x/mo for 3mos

Visit 1x/mo for 3mos

Referral Request Review Suggest Teacher


17/90

TeachStreet: 45-day Claim Rates(Seattle, Portland)


18/90

Role: Product / Engineering

Q: What Features to Build? Why? When are you Done?

A: Easy-to-Find, Fun/Useful, Unique Features that

Increase Conversion (stop iterating when increase decelerates)

Wireframes = Conversion Steps

Measure, A/B Test, Iterate FAST (daily/weekly)

Optimize for Conversion Improvement

80% on existing feature optimization

20% on new feature development


19/90

Example Conversion Dashboard(note: *not* actuals your mileage may vary)Stage Conversion Status Conv.

%

Est. Value

(*not* cost)

Acquisition Visitors -> Site/Widget/Landing Page(2+ pages, 10+ sec, 1+ clicks = dont abandon)

60% $.05

Activation Happy 1st

Visit; Usage/Signup(clicks/time/pages, email/profile reg, featureusage)

15% $.25

Retention Users Come Back; Multiple Visits(1-3x visits/mo; email/feed open rate / CTR)

5% $1

Referral Users Refer Others

(cust sat >=8; viral K factor > 1; )

1% $5

Revenue Users Pay / Generate $$$(first txn, break-even, target profitability)

2% $50


20/90

Role: Marketing / Sales

Q: What channels? Which users? Why?A: High Volume (#), Low Cost ($), High Conv (%)

Design & Test Multiple Marketing Channels + Campaigns Select & Focus on Best-Performing Channels & Themes Optimize for conversion to target CTAs, not just site/landing page Match/Drive channel cost to/below revenue potential

Low-Hanging Fruit: Blogs

SEO/SEM Landing Pages Automated Emails


21/90

Example Marketing Channels

PR

Contest

Biz Dev Direct Marketing

Radio / TV / Print

Dedicated Sales

Telemarketing

Email

SEO / SEM

Blogs / Bloggers

Viral / Referral

Affiliate / CPA

Widgets / Apps

LOLCats ;)
http://www.flickr.com/photos/500hats/2236563985/in/set-72157600787878946


22/90

MAARRRketing PlanMarketing Plan = Target Customer Acquisition Channels

3 Important Factors = Volume (#), Cost ($), Conversion (%)

Measure conversion to target customer actions

Test audience segments, campaign themes, Call-To-Action (CTAs)

[Gradually] Match Channel Costs => Revenue Potential Increase Vol. & Conversion, Decrease Cost, Optimize for Revenue Potential

Avg Txn Value (ATV), Ann Rev Per User (ARPU), Cust Lifetime Value (CLV)

Design channels that (eventually) cost


23/90

One Step at a Time.

1. Make a Good Product: Activation & Retention

2. Market the Product: Acquisition & Referral

3. Make Money: Revenue & Profitability

You probably cant save yourAss and yourFace

at the same time choose carefully. DMC


24/90

Summary

Measure Stuff. Keep It Simple.

5 Steps: Startup Metrics for Pirates (AARRR!)

3 Items: Biz Model, Conversion, Mktg Channels Iterate & Optimize. Feedback Loop. (Loop. Loop).

One Step at a Time.


25/90

Links & Resources

Additional References:

Influence: The Psychology of Persuasion, Robert Cialdini (book)

Putting the Fun in Functional, Amy Jo Kim (etech 2006 preso)

Futuristic Play, Andrew Chen (blog)

Dont Make Me Think, Steve Krug (book)

Designing for the Social Web, Joshua Porter (book, website)

Startup Lessons Learned, Eric Ries (blog)

Customer Development Methodology Steve Blank (presentation, blog)

Startup-Marketing.com, Sean Ellis (blog)
http://www.amazon.com/Influence-Psychology-Persuasion-Robert-Cialdini/dp/0688128165http://www.slideshare.net/amyjokim/putting-the-fun-in-functionahttp://andrewchenblog.com/http://www.amazon.com/Common-Approach-Usability-Circle-Com-Library/dp/0789723107http://www.amazon.com/Common-Approach-Usability-Circle-Com-Library/dp/0789723107http://bokardo.com/archives/designing-for-the-social-web-the-book/http://startuplessonslearned.blogspot.com/http://www.slideshare.net/venturehacks/customer-development-methodology-presentation/http://steveblank.com/http://startup-marketing.com/http://startup-marketing.com/http://startup-marketing.com/http://startup-marketing.com/http://steveblank.com/http://www.slideshare.net/venturehacks/customer-development-methodology-presentation/http://startuplessonslearned.blogspot.com/http://bokardo.com/archives/designing-for-the-social-web-the-book/http://www.amazon.com/Common-Approach-Usability-Circle-Com-Library/dp/0789723107http://www.amazon.com/Common-Approach-Usability-Circle-Com-Library/dp/0789723107http://andrewchenblog.com/http://www.slideshare.net/amyjokim/putting-the-fun-in-functionahttp://www.amazon.com/Influence-Psychology-Persuasion-Robert-Cialdini/dp/0688128165


26/90

Dan OlsenDan OlsenCEO,CEO,YourVersionYourVersion

Web 2.0 ExpoWeb 2.0 ExpoApr 3, 2009Apr 3, 2009How to Use the Force ofHow to Use the Force of

Metrics to Optimize yourMetrics to Optimize your


27/90

The Force of Metrics is Real

The Force of Metrics?Really?


28/90

Copyright 2009 Olsen Solutions LLCCopyright 2009 Olsen Solutions LLC

Business as an OptimizationBusiness as an Optimization

ExerciseExercise

Given reality as it exists today,Given reality as it exists today,

optimize our resultsoptimize our results

subject to our resource constraints.subject to our resource constraints.


29/90

Profit =Profit = RevenueRevenue - Cost- Cost

Unique VisitorsUnique Visitors xx Ad Revenue per VisitorAd Revenue per Visitor

Impressions/VisitorImpressions/Visitor x Effective CPM / 1000x Effective CPM / 1000

Visits/Visitor x Pageviews/Visit xVisits/Visitor x Pageviews/Visit xImpressions/PVImpressions/PV

New VisitorsNew Visitors + Returning Visitors+ Returning Visitors

Invited VisitorsInvited Visitors + Uninvited Visitors+ Uninvited Visitors

# of Users Sending Invites x Invites Sent/User x Invite# of Users Sending Invites x Invites Sent/User x InviteConversion RateConversion Rate


BusinessBusiness

Peeling the OnionPeeling the Onion


30/90

Create ratios from primary metrics: X / YCreate ratios from primary metrics: X / Y Example:Example: How good is your registration page?How good is your registration page? Okay:Okay: # of registered users per day# of registered users per day Better:Better: registration conversion rate =registration conversion rate =

# registered users / # uniques to reg page# registered users / # uniques to reg page

Need third-party analytic tools PLUS home-grownNeed third-party analytic tools PLUS home-grownmetricsmetrics

Track each metric as daily time seriesTrack each metric as daily time series


How to Track Your MetricsHow to Track Your Metrics

DateDateUniqueUniqueVisitorsVisitors

PagePageviewsviews

AdAdRevenueRevenue

New UserNew UserSign-upsSign-ups

4/24/084/24/08 10,10010,100 29,60029,600 2525 490490

4/25/084/25/08 10,50010,500 27,10027,100 2424 480480


31/90


Ideas

Importance and Satisfaction To offer a good customer value

proposition:Meet High Importance needs with HighSatisfaction

98

8784

8679 847055 80

7280

75

41

50

55

60

65

70

75

80

85

90

95

100

40 50 60 70 80 90 100

Satisfaction

Importance

See Anthony UlwicksWhat Customers Want

BadBad

GreaGreatt

Prioritizing Product Ideas byPrioritizing Product Ideas by


32/90


Prioritizing Product Ideas byPrioritizing Product Ideas byROIROI

Investment (developer-

weeks)

Retur

n

(Value

Crea

ted)

Idea C

Idea B

Idea D

Idea A

Idea F

1

1

2 3 4

2

3

4?

Marrying UI Design with


33/90

Marrying UI Design withMetrics

Think of Ease ofuse as distinctfrom thefunctionality

User Interface

elements matter Position Layout Size

Color FontText copy Interaction design

Navigation

Improving Signup PageImproving Signup Page


34/90


Improving Signup PageImproving Signup PageConversionConversion

New Registere

90%

100%

Improving SignImproving Sign


35/90


Open

Account

Sign in

Account

Selection

Register

56%

44%

ForgetPassword

Registration

Process

45% drop off(20% oftotal)

36%overalldrop offfor this

step

70%(32% ofTotal)

17% dropoff (10% of

total)

20% dropoff

(6% of total)

30%(14% of Total)

80%(26% ofTotal)

55%(24% of Total)

64%of Total

Improving SignImproving SignIn/Registration FlowIn/Registration Flow

ChangePassword

83%(46% of Total)


36/90

Abandonment Rate (7 Day Moving Average)

0%

10%

20%

30%

40%

50%

60%

70%

80%

10/7/02

10/14/02

10/21/02

10/28/02

11/4/02

11/11/02

11/18/02

11/25/02

12/2/02

12/9/02

12/16/02

12/23/02

12/30/02

1/6/03

1/13/03

1/20/03

Aba

ndonmentRate(7DayMoving

Average)

Steps 1-2


Improved RegistrationImproved Registration

Conversion RateConversion Rate

37% improvementin conversion rate

ReleasedNew Design

Optimization through Iteration:Optimization through Iteration:


37/90


Optimization through Iteration:Optimization through Iteration:Continuous ImprovementContinuous Improvement

Measurethe metric

Analyzethe metric

Identify topopportunitiesto improve

Design &develop theenhancement

Launch theenhancement

Learning

Gaining knowledge:

Market

Customer

Domain

Usability

Summary: Using the Force ofSummary: Using the Force of


38/90


Summary: Using the Force ofSummary: Using the Force ofMetricsMetrics

Define what success meansDefine what success means Equation of your businessEquation of your business

Customer value propositionCustomer value proposition

Instrument your site and track keyInstrument your site and track keymetricsmetrics

Identify opportunities and prioritizeIdentify opportunities and prioritize

by ROIby ROI

Launch, learn, and iterateLaunch, learn, and iterate


39/90

[email protected]@yourversion.com

@danolsen@danolsen

www.yourversion.comwww.yourversion.comMay the ForceMay the Forceof Metrics beof Metrics be

with you!with you!
http://www.yourversion.com/


40/90

Dogster, Inc. Case StudyTed Rheingold

blog.dogster.com / @tedr

How actionable metrics informed uswe were chasing our tail the wrongkey results yet still guided us to the

primary objective bacon makingmoney


41/90


42/90

Dogster, Inc.

Sites & Services

Dogster.com, Catster.com, Snuzzy.com

Togethertag.com

5 year old profitable business. No VC, Angel only

Primary Revenue Source

Directly sold ad inventory

Secondary Sources

User subscriptions

Virtual currency

Together Tag (Physical dog & cat ID tags w/ always-on web connectingservice)


43/90

Marketing Initiative Feb 07

Goal: 2,000 New Registrants a Day

Marketed sites via AdSense and BlogAds

At launch we linked ads to our homepages

Quickly made a general landing page with introinfo and big link to register

Then made dynamic landing with relevant textand registration form.

This significantly increased percentage of ad

clickers that became registrants


44/90

LANDING PAGE Welcome


45/90

Landing Page Cute Overload


46/90

LANDING PAGE Daily Candy


47/90

LADING PAGE Dog Info


48/90

REGISTRATION PAGE

-Activate


49/90

Quantitative Success DoesntMean Qualitative Success

We hit key result of 2k registrants a day

BUT only 60% were activatingOnly 8% were making profile pages vs. 80%that found site on their own

Real members were generating a $4 ARPU,these were worth $0.10


50/90

Actionable Results

Within 3 months we halted all ad spending.

We were sitting on a treasure trove of what topicsengaged pet people the most.

We could drive a lot of uniques to our sites andadvertisers love uniques


51/90

HOMEPAGE - Current


52/90

CRAZY EGG Searchers


53/90

Dogster Search Traffic


54/90


55/90

Questions? #metrics@davemcclure @danolsen

tedr


56/90

Appendix


57/90

Startup Metrics

Activation

A ti ti


58/90

Website.com

Activation Criteria:

10-30+ seconds

2-3+ page views

3-5+ clicks

1 key feature usage

do LOTS oflanding page& A/B tests

make lots of dumbguesses & iterate FAST

SEOSEM

Apps &Widgets

Affiliates

Email

PR BizDev

Campaigns,Contests

Direct,Tel, TV

SocialNetworksBlogs

Domains

Activation


59/90

ActivationWhat do users do on their first visit?

Example Activation Goals

Click on something!

Account sign up / Emails

Referrals / Tell a friend

Widgets / Embeds

Low Bounce Rate

Activation Tips

Less is more Focus on user experience / usability

Provide incentives & call to actions

Test and iterate continuously


60/90

ActivationWhat do users do on their first visit?

Key Metrics to TrackPages per visit

Time on site

Conversions


61/90

ActivationTools

Crazy Egg (Visual Click Mapping)http://crazyegg.com

Google Website Optimizer (A/B & Multivariate Testing)

http://google.com/websiteoptimizer

Marketo.com (B2B Lead Generation Management)

http://marketo.com

ResourcesExperimentation and Testing: A Primer

kaushik.net/avinash/2006/05/experimentation-and-testing-a-primer.html

Landing Page Design Toolbox: 100 Tips & Toolshttp://tinyurl.com/326co6

Landing Page Tutorials & Case Studieshttp://www.copyblogger.com/landing-pages/

101 Easy Easy to use Google Website Optimizer

http://conversion-rate-experts.com/articles/101-google-website-optimizer-tips/
http://crazyegg.com/http://crazyegg.com/http://google.com/websiteoptimizerhttp://google.com/websiteoptimizerhttp://marketo.com/http://marketo.com/http://www.kaushik.net/avinash/2006/05/experimentation-and-testing-a-primer.htmlhttp://tinyurl.com/326co6http://www.copyblogger.com/landing-pages/http://www.copyblogger.com/landing-pages/http://conversion-rate-experts.com/articles/101-google-website-optimizer-tips/http://conversion-rate-experts.com/articles/101-google-website-optimizer-tips/http://www.copyblogger.com/landing-pages/http://www.copyblogger.com/landing-pages/http://www.copyblogger.com/landing-pages/http://tinyurl.com/326co6http://www.kaushik.net/avinash/2006/05/experimentation-and-testing-a-primer.htmlhttp://marketo.com/http://marketo.com/http://google.com/websiteoptimizerhttp://google.com/websiteoptimizerhttp://crazyegg.com/http://crazyegg.com/


62/90

Startup Metrics

Retention


63/90

R t ti


64/90

Cohort Analysis:

Distrib of Visits over Time

Rate of Decay

Effective Customer Lifecycle

Retention Methods

Automated Emails* Track open rate / CTR / Quantity

RSS / News Feeds* Track % viewed / CTR / Quantity

Widgets / Embeds* Track impressions / CTR / Quantity

RetentionHow do users come back? How often?


65/90

Example Retention Goals

1 - 3+ visits per month

20% open rate / 2% CTR

High deliverability / Low spam rating

Long customer life cycle / Low decay

Identify fanatics and cheerleaders

Retention Tips

Email is simple and it works BUT make unsubscribe easy

80% subject line / 20% body text

ACTUALLY 99% subject line / 1% body text

Fanatics = virality + affiliate channel (bloggers?)



66/90


Key Metrics to Track Source Quantity

Conversions Visitor Loyalty Session Length


67/90


68/90

Startup Metrics

Acquisition

A i iti


69/90

Website.com

Marketing Channels:

largest-volume (#) lowest-cost ($) best-performing (%)

Acquisition

SEOSEM

Apps &Widgets

Affiliates

Email

PR BizDev

Campaigns,

Contests

Direct,Tel, TV

SocialNetworks

Blogs

Domains

Acquisition


70/90

AcquisitionWhere are users coming from?

Acquisition Methods

SEO / SEM

Blogs

Email

Social Media &

Social Networks

Domains

Acquisition


71/90

AcquisitionKeyword Vocabulary

Top 10 - 100 words

Your Brand / Products

Customer Needs / Benefits Competitors Brand / Products

Semantic Equivalents

Misspellings

Things to analyze

Sources

Volume

Cost

Conversion


72/90


73/90

AcquisitionTools

Google Analytics (web analytics)google.com/analytics

Google Keyword Tool (keyword research tool)

adwords.google.com/select/KeywordToolExternal

SEO Book Tools (SEO related tools)tools.seobook.com

ResourcesSEO Book Blog

seobook.com/blog

The Social Media Manual: Read Before You Playsearchengineland.com/071120-144401.php

Strategies to ruthlessly acquire users

andrewchen.typepad.com/andrew_chens_blog/2007/04/10_obvious_stra.html


74/90

Startup Metrics

Referral

Referral


75/90

Website.com

Acquisition

SEOSEM

Apps &Widgets

Affiliates

Email

PR BizDev

Campaigns,Contests

Direct,Tel, TV

SocialNetworks

Blogs

Domains

Retent

ion

Emails &Alerts


Features


Focus on driving referrals*after* users have ahappy experience;avg score >= 8 out of 10

e e a


76/90

ReferralHow do users refer others?

Referral Methods

Send to Friend:

Email / IMSocial Media

Widgets / Embeds

Affiliates


77/90

ReferralViral Growth Factor

Viral Growth Factor = X * Y * ZX = % of users who invite other people

Y = average # of people that they invited

Z = % of users who accepted an invitation

A viral growth factor > 1 means an exponential organic user acquisition.


78/90

ReferralTools

Gigya (social media distribution & tracking tool)gigya.com

ShareThis / AddThis (sharing buttons)

sharethis.com / addthis.com

GetMyContacts (PHP contacts importing & invitation software)getmycontacts.com

ResourcesSeven Ways to GO VIRAL

lsvp.wordpress.com/2007/03/02/seven-ways-to-go-viral/

Whats your viral loop? Understanding the engine of adoptionandrewchen.typepad.com/andrew_chens_blog/2007/07/whats-your-vira.html

Metrics: Where Users Come From

slideshare.net/guest2968b8/rockyou-snap-summit-32508


79/90

Startup Metrics

Revenue

Revenue


80/90

Website.com

Revenue$$$

Biz DevAds, Lead Gen,Subscriptions,ECommerce

Acquisition

SEOSEM

Apps &Widgets

Affiliates

Email

PR BizDev

Campaigns,Contests

Direct,Tel, TV

SocialNetworks

Blogs

Domains

Retent

ion

Emails &Alerts


Features


This is the part *you*still have to figure out(we dont know jack

about your business)

Revenue


81/90

RevenueHow do you make money?

Revenue Tips

Dont Rely on AdSense (only) Start Free => 2% Freemium

Subscription / Recurring transactions

Qualify your customers -> Lead generation(arbitrage)

Sell something! (physical or virtual)


82/90

RevenueResources & Tools

Revenue Metrics (Andrew Chen)http://tinyurl.com/47r63a

How to Create a Profitable Freemium Startup (Andrew Chen)

http://tinyurl.com/8z9ygk

2008 Affiliate Marketing Review (Scott Jangro)

http://tinyurl.com/86wak4

Be Bold Be Humble
http://tinyurl.com/47r63ahttp://tinyurl.com/8z9ygkhttp://tinyurl.com/86wak4http://tinyurl.com/86wak4http://tinyurl.com/8z9ygkhttp://tinyurl.com/47r63a


83/90

Be Bold. Be Humble.

Stuff That Matters:

Passion for Problem/Solution + Hypothesis ofCustomer Lifecycle

1-page Business Model: Prioritized List of (Users + Conversions)

Critical, Few, Actionable Metrics + Dashboard of Measured User Behavior 1-pg Marketing Plan: (Channels + Campaigns) * (Volume, Cost, Conv %)

Velocity of (Product Execution + Cycle Time ofTesting) * Iteration

20% Inspiration+ 80% Perspiration

Long-term: Audacity + Creative Inspiration

Short-term: Humility + Analytic Rigor

Passion vs Precision


84/90

Passion vs. Precision

Solve a Problem = Passion Optimize the Solution =Precision

BUT:

Precision is Illusion; data is fuzzy

Collecting & Interpreting data takes time & effort

Keep Metrics Simple & Actionable

Long-term: Audacity + Creative Inspiration

Short-term: Humility + Analytic Rigor

Types of Measurement


85/90

Types of Measurement

1. Qualitative: Usability Testing / Session Monitoring

Watch users, guess problems & solutions from small # of users

1. Quantitative: Traffic Analysis / User Engagement

Track users, usage, conv %'s for empirical sample # of users

1. Comparative: A/B, Multivariate Testing

Compare what users do in one scenario vs another

Measure which copy/graphics/UI are most effective

1. Competitive: Monitoring & Tracking Competitors Track competitor activity & compare against yours (if possible)

Compare channels, keyword traffic, demographics, user sat, etc.

Design Optimize for Conversion


86/90

Design, Optimize forConversion

Kano Model: User Needs &S ti f ti


87/90


SatisfactionUserUser

SatisfactionSatisfaction

UserUser

PerformancePerformance(more is(more isbetter)better)

DelighterDelighter(wow)(wow)

NeedNeednotnotmetmet

NeedNeedfullyfullymetmetMust HaveMust Have

Needs &Needs &featuresfeatures

migrate overmigrate overtimetime

Needs


88/90


Is the site up when I want to use it?

Is the site fast enough?

Does the functionality work?

Does the functionalitymeet my needs?

Needs(adapted from Maslow)

Customers Perspective What does it mean to us?

Uptime

Page Load Time

Absence of Bugs

Feature Set

Usability & Design

Decreasing

Dissatisfaction

Increasing

Satisfaction

How easy to use is it?

Optimization to your Product


89/90


Optimization to your ProductProcess

PlanPlan

DesigDesig

nn

DevelopDevelop

BusinessObjectiv

es

ProductObjectiv

es

PrioritizedFeature List

Scoping

Requirements & Design

Code Test Launch

SiteLevel

Feature Level

OptimizeOptimize Metrics &UserFeedback

Using Metrics to Optimize theUsing Metrics to Optimize theE ti f B iE ti f B i


90/90

Equation of your BusinessEquation of your Business What are the metrics for your business?What are the metrics for your business?

Where is current value for each metric?Where is current value for each metric? How many resources to move eachHow many resources to move each

metric?metric? Developer-hours, time, moneyDeveloper-hours, time, money

Which metrics have highest ROIWhich metrics have highest ROIopportunities?opportunities?

Retu

r

Retu

r

nn

InvestmentInvestment

Retu

r

Retu

r

nn


Retu

r

Retu

r

nn


Metric AMetric AGood ROIGood ROI

Metric BMetric BBad ROIBad ROI

Metric CMetric CGreat ROIGreat ROI

How to Be a Web 2.0 Metrics Jedi - Web 2.0 Expo (Apr 09)

Documents