KPIs & metrics (slides by Matteo Panfilo) Presenters [email protected] @mattepv [email protected] @umbertodefeo GiT Workshop - January 2014
Jan 27, 2015
KPIs & metrics (slides by Matteo Panfilo)
Presenters
@mattepv
@umbertodefeo
GiT Workshop - January 2014
About us
1
What I’ve done until 2013 I studied economics in Pavia & management in France I wrote a thesis on Angel Investing I attended the AIFI Class for VC professionals in 2011 I worked over two years @ Italian Angels for Growth (11 investments in the period = >4mln)
I worked one year @ Fashionis (e-commerce startup) What I'm doing since the beginning of 2014 I co-founded @Fast4ward I keep on promoting local startup ecosystems @ Activators Pavia
What I’ve done until 2013
I studied International Business and Finance at LUISS I wrote a thesis on the investment opportunities in the green energy sector in South East Asia I lived and worked in Kiev, London, Rome, Kuala Lumpur and Milan I attended the AIFI Class for VC professionals in 2012 I worked @ Italian Angels for Growth
I worked @ Zernike META Ventures What I'm doing since the beginning of 2014 I started working on the international activities @META Group
Agenda
1.Metrics & KPIs
2. Funnel
3.Cohort analysis
4.Churn rate
5.A/B test
6.Source channels, email/DEM
7.CRM
Conclusion
2
Agenda
1.Metrics & KPIs
2. Funnel
3.Cohort analysis
4.Churn rate
5.A/B test
6.Source channels, email/DEM
7.CRM
Conclusion
3
1. Different businesses, different metrics
Airbnb
Friendfinder
Blogo
6sicuro.it
Privalia
4
Number of house with HD pics
Viral factor (K)
Pages per user
Generated quotes (lead)
Cost per shoting
Hundreds of different metrics
PV Page Views
UV Unique Visitors
CPM Cost per Thousand
BR Bounce Rate
CTR Click Trough Rate
CPR Cost Per Registration
CPL Cost Per Lead
CPA Cost Per Acquisition (or
Cost per Action)
CPC Cost Per Click
LTV Lifetime Value
CR Conversion Rate
Churn Churn Rate
5
Lean Analytics
6 http://www.startuplessonslearned.com/2013/03/lean-analytics.html
http://www.slideshare.net/Leananalytics
http://www.kaushik.net/avinash/lean-analytics-cycle-metrics-hypothesis-experiment-act/
Lean Analytics (alcuni esempi)
7 http://www.startuplessonslearned.com/2013/03/lean-analytics.html
http://www.slideshare.net/Leananalytics
http://www.kaushik.net/avinash/lean-analytics-cycle-metrics-hypothesis-experiment-act/
Scalability (basic formulas)
LTV
CPC
%CR
CPA = Cost Per Acquisition (or Cost per Action), CPC = Cost per Click, LTV = Lifetime Value
LTV Starbucks: http://blog.kissmetrics.com/wp-content/uploads/2011/08/calculating-ltv.pdf
LTV Avinash Kaushik: http://www.kaushik.net/avinash/analytics-tip-calculate-ltv-customer-lifetime-value/
Dipende fondamentalmente
da: soddisfazione dei clienti, CRM, biz model, etc
< CPA
8
“LTV is the projected profit that a
customer will generate during his
lifetime”
General CPA = $ Adv cost / # Customers acquired CPA based on CPM campaign: (CPM x Impressions /1000)/ # Customer acquired LTV formulas
•LT Revenues: # Purchases x $ Avg. basket •LT Revenues: $ Avg. basket / % Churn rate •LT Gross Profit: # Purchases x $ Avg. basket x % gross profit margin •LT Gross Profit: $ Avg. basket / % Churn rate x % gross profit margin •LT DCF: ($ Avg. basket margin x Life Expectancy) x (Customer retention rate / 1 + discount rate – Customer retention rate)
Scalability (basic formulas)
LTV
CPC
%CR
< CPA
9
Small numbers,
huge impact:
CPC = 0,4€
%CR = 4%
CPA = 10€
CPC = 0,4€
%CR = 2,5%
CPA = 16€
CPA = Cost Per Acquisition (or Cost per Action), CPC = Cost per Click, LTV = Lifetime Value
LTV Starbucks: http://blog.kissmetrics.com/wp-content/uploads/2011/08/calculating-ltv.pdf
LTV Avinash Kaushik: http://www.kaushik.net/avinash/analytics-tip-calculate-ltv-customer-lifetime-value/
Scalability, make it happen!
10 Source: David Skok - http://www.forentrepreneurs.com/startup-killer/
Note: CAC = CPA
Source: David Skok - http://www.forentrepreneurs.com/startup-killer/
Scalability, how you can do it?
Note: CAC = CPA 11
Conversion
What’s the meaning of conversion rate?
What’s the meaning of funnel?
CPC
%CR CPA =
Define what are you looking to measure! i.e.: define who you consider a customer; define the difference between
CPL and CPA (if there is)
12
General CPA = $ Adv cost / # Customers acquired CPA based on CPM campaign: (CPM x Impressions /1000)/ # Customer acquired
The Funnel .. <offline mode>
13
1. A lot of people see the adv
wall
2. Some people recognize it
walking in front of a
sunglasses shop
3. Some of them decide to go
in and try them
1. One of them want to buy it
The Funnel .. <online mode - ie. ecommerce>
100
50
20
5
3
1
1. I paid to have some users on my
website
2. They arrive on the landing page
3. They leave their email
1. They put one product in the basket
2. They start the checkout process
3. They complete all the payment
info
4. They pay
50%
40%
40%
38%
33% 1%
14
The Funnel .. <online mode - ie. ecommerce>
100
50
20
5
3
1
Paid users
Landing page
Leave the email
Product on the basket
Checkout process
Payment info
They pay
• Pictures
• Copy/text button
• Info required
• Password
• Email confirm
• Add button
• Related product
• Info required
• Shipping days
• Delivery cost
• Payment gateway
• Invoice/legal info
• CC circuit
• PIN request
Some drivers:
15
Why measure the funnel
You need to identify the bottleneck and solve it!
Examples:
- Pictures
- Checkout process
- Payment gateway 16
The Funnel .. (flash sales ecommerce)
Users that arrive till
checkout
Registered users
Unique users
Users that complete the form
Paying
users
1%-0,5%
Measure your CPL/CPA!
The CPA initially refers only to
the traffic resulting from paid
campaigns is the main KPI to
be monitored in an
ecommerce / SaaS, but in
the medium term it should
be monitored along with the
metrics of the traffic that
came from other channels
(email, facebook page, etc.
)
17
Customer journey
Use
rs
Cu
stom
ers
Source: IAB Seminar 2013, Ilaria Zampori Managing Director QUISMA, Individual Conversion Attribution
http://www.iabseminar.it/wp-content/uploads/2013/02/9_QUISMA.pdf
Awareness
Consideration Conversion
Loyalty
Choiche Information
Desire
18
Customer journey
Awareness
Consideration Conversion
Loyalty
Source: IAB Seminar 2013, Marco Caradonna Simple Agency Dynamic Customer Journey
http://www.slideshare.net/mcarado/4-iab-seminar-performance-2013-simple-agency
Choiche Information
Desire
19
instruments instruments instruments
Agenda
1.Metrics & KPIs
2. Funnel
3.Cohort analysis, measure customer behaviour
4.Churn rate
5.A/B test
6.Source channels, email/DEM
7.CRM
Conclusion
20
Cohort Analysis referral source analysis
“A cohort is a group of people who share a common
characteristic over a certain period of time”
Google Facebook Blog
Customer spending by referral source
Direct
Source: www.cohortanalysis.com/
21
Cohort Analysis time-period analysis
Time period
1 2 3 4 5 6 7 8 9 10 11 12
Time-period cohorts
1 513 410 328 263 210 168 134 108 86 69 55 44
2 923 739 591 473 378 303 242 194 155 124 99
3 1.258 1.007 805 644 515 412 330 264 211 169
4 1.537 1.230 984 787 630 504 403 322 258
5 1.777 1.421 1.137 910 728 582 466 373
6 1.987 1.590 1.272 1.017 814 651 521
7 2.178 1.742 1.394 1.115 892 714
8 2.355 1.884 1.507 1.206 965
9 2.524 2.020 1.616 1.292
10 2.689 2.151 1.721
11 2.852 2.282
12 3.017
paying users 513 1.334 2.325 3.398 4.495 5.583 6.644 7.671 8.661 9.618 10.547 11.454
How much revenue do the users that I’ve acquired on the second months
generate?
How many users that I’ve acquired on the second month will I probably still
have on the ninth month?
Source: andrewchenblog.com
22
Agenda
1.Metrics & KPIs
2. Funnel
3.Cohort analysis
4.Churn rate, do not throw the job done
5.A/B test
6.Source channels, email/DEM
7.CRM
Conclusion
23
Churn Rate
Your churn rate is the amount of customers or subscribers who cut ties
with your service or company during given time period.
These customers have “churned”
Recurring revenues model Traditional Ecommerce
# Subscribers lost
# Starting subscribers = % Churn rate
# Customers who has not made a
purchase in 90 days*
# Starting active customers
= % Churn rate
Source: www.churn-rate.com
24
* it depend on
business model
and offer
Optimize..
1.Metrics & KPIs
2.Funnel
3.Cohort analysis
4.Churn rate
25
Analyse
Change
Learn Consolidate
Iterate Learning
process
Why you need to optimize..
• You have 0-8 seconds to make a compelling headline and
landing page. After 8 seconds, the majority of visitors leave.
• Approximately 96% of visitors that come to your website are
not ready to buy.
• The more landing pages you have, the more leads you are
likely to get.
• Product videos can increase purchases of the product by
144%.
• A 1 second delay in your site speed can result in a 7%
reduction in conversions.
• A/B testing is becoming the preferred method that has
brought a lot of the companies the most success.
Source: http://blog.kissmetrics.com/what-converting-websites-do/ 26
Agenda
1.Metrics & KPIs
2. Funnel
3.Cohort analysis
4.Churn rate
5.A/B test, what do customers prefer?
6.Source channels, email/DEM
7.CRM
Conclusion
27
A/B test…
• Elaborate 2 different versions (text, colors, font, layout, etc)
• Propose versions to the target sample (with the same content)
• Measure conversion rate
• Profile users based on conversion rates (male/female, addicted, etc)
• (Visualize where the users click on the pages)
The same process could be applicable to web pages, email/DEM,
games dynamics, etc!
28
Headline..
Original
+30%
+15%
29 Fonte: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page
Layout..
30 Fonte: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page
Layout..
31 Fonte: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page
Layout..
Is not always a good
idea to write more…
32 Fonte: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page
A/B test are easy but..
Source: http://techcrunch.com/2013/01/12/current-conversion-rate-and-desired-
confidence-interval-will-help-you-avoid-analysis-paralysis-stop-running-stupid-tests/
Consider a site that has 10,000 visitors per month and has a 5 percent conversion
rate. The table below shows how long it will take to run a “conclusive” test (95
percent confidence) based on how much the change impacts conversion rate.
33
Other tips..
• Badging
up to +50%
• Message consistency
up to +100%
• International website
up to +100%
• Email sign-up
up to +1.000%
Fonte: http://www.slideshare.net/Emerce/emerce-performance-bryan-eisenberg 34
Agenda
1.Metrics & KPIs
2. Funnel
3.Cohort analysis
4.Churn rate
5.A/B test
6.Source channels, email/DEM
7.CRM
Conclusion
35
Email & DEM
1. Choose & Optimize email form
2. Choose & Optimize password request
3. Choose the confirmation process
4. Choose emails’ arguments and style
5. Choose & Optimize email subjects
6. Metrics, metrics, metrics:
• Open rate/Unique OR
• Click rate
• Click-through-rate
• Delivery time
• List growth
• Unsubscribe rate (or churn rate)
• Spam complaint rate
Drivers:
Subject, time
Content
Layout
All of the above
Too many
Frequency & content
Too many
Email could become your first traffic source, improve the LTV of your
customers, decrease your customer acquisition cost !
36
Email & DEM
Source: http://retail-analytics.quora.com/Email-Open-Click-Through-Analytics-from-over-80-Billion-Emails
37
Agenda
1.Metrics & KPIs
2. Funnel
3.Cohort analysis
4.Churn rate
5.A/B test
6.Source channels, email/DEM
7.CRM
Conclusion
38
The true cost of bad support
39 Source: https://www.helpscout.net/blog/bad-customer-service/
Interact with customers…
People now expect the same—if not better— level of
service from online storefronts and service channels as
they do from a visit to a retail store or a phone
conversation with an agent.”
Source: “ E mail Customer Service in North American Small and Medium Businesses” by
BenchmarkPortal, 2005 and https://www.helpscout.net/blog/bad-customer-service/ 40
Source: Communication Preference, Live Chat Effectiveness, A Survey of Internet Shoppers 2012
Interact with customers…
41
Source: “Email Customer Service in North American Small and Medium Businesses” by BenchmarkPortal, 2005,
Source: Forrester Research Inc., 2008
83% of small and medium-sized
businesses responded with
inaccurate or incomplete answers.
41% of consumers expect an e-mail
response within six hours.
Only 36% of retailers responded that quickly.
Interact with customers…
42
“Customer experience quality could
result in a swing of $184 million for a
large Internet retailer.”
Source: Forrester Research Inc., 2008
Interact with customers…
43
The power of great video…
Video: http://www.youtube.com/watch?feature=player_embedded&v=ZUG9qYTJMsI
44
Agenda
1.Metrics & KPIs
2. Funnel
3.Cohort analysis
4.Churn rate
5.A/B test
6.Source channels, email/DEM
7.CRM
Conclusion
45
Recap & useful tools
1.Metrics, measure and understand your KPIs
2.Funnel, know the steps to improve your conversion
3.Cohort analysis, analyse customer behaviour
4.Churn rate, do not throw the job done
5.A/B test, what do customers prefer?
6.Source channels, email/DEM still a very interesting
channel
7.CRM, interact and learn from your customers
- Mixpanel
- Jirafe free
- Woopra
- Kissmetrics
- RJ metric
- Geckoboard
46
My 2 cents
Non pensate che fare una startup sia un gioco.
Cercate di creare valore, non fate le cose perché vanno di moda
e non sviluppate l’ennesima app per condividere le foto.
Non pensate a raccogliere soldi ma pensate a creare un prodotto
e ad avere una visione per i successivi 10 anni.
Studiate e documentatevi tanto prima di prendere ogni decisione,
c’è un’enorme quantità di informazioni online, basta saperle cercare.
47
“I believe in the power of a well-operated, sophisticated organization
that generates social and economic value. My career goal is to found
and lead this type of organization.” Hansoo Lee
• Steve blank tools (www.steveblank.com/tools)
• Avinash Kaushik (www.kaushik.net)
• Quora (follow Matthew Caroll)
• Slideshare (xerox, 500startups, etsy)
• Techcrunch
• Angel.co
• Italian Startup Scene
• Indigeni Digitali
• Startupnumbers.com ;)
48
Must read/view:
• Blog etsy (http://mcfunley.com/)
• Blog Fabrice Grinda
• Blog fab.com
• Blog google
• Blog a16z.com
• Foundat.io/n
• ecorner.stanford.edu
• avc.com
• Angel.co
Must read/view:
49
Primo esempio: wikli
+70%
Spesa tot: 200€ Strumenti utilizzati: opimizely
Utenti attuali >6.000
Secondo esempio..
Secondo esempio…iterazione
Secondo esempio…iterazione
Terzo esempio…non solo web
Quarto esempio (fashion subscription commerce)
Conversion rates: From inception October 30th until December 31th
• Unique visitors to registered free members: 27% Benchmark successful players: circa 22%
• Free members to paying subscribers: 2,6% in 2 mesi Benchmark successful players: circa 3% in un anno
Absolute metrics:
Form inception November 5th until December 30th
•26k unique visitors
•7k registered members
•200 subscription sales
•7k $ revenues
Traffic sources: •Direct: 59%
•Facebook 23%
•Daily Deal Sites 10%
•Others 9%
Always in test..
Video (da vedere!): http://ecorner.stanford.edu/authorMaterialInfo.html?mid=2313
“We put it in a feature that we can
build in a week. It's a ghetto build. We
AB test it. We flow test it. We put it out
to 1%, 10% of our users. We build a
data warehouse with the testing
platform. So, we're running several
hundred tests at any given time for
every one of our games. And no single
user ever is involved in more than one
test. So, we love testing. When we see
that it moves our metrics in a
considerable way, that's when we take
it to be a full feature roll out and then
we do the full 2.0.
Capitalizzazione 3,6 $bn --- Ricavi 1,22 $bn --- 12% ricavi da facebook
Always in test..
Sources: http://ecorner.stanford.edu/authorMaterialInfo.html?mid=2313
“We don't want to go down those paths until we've test them
with our users. And what's amazing and this is a feature that you will all have available to you as you enter this third Internet
business is that you're going to run a service and you are going
to every week be able to test things with your users and
something that I never have available to me in previous companies. It won't be build it for three months and hope and
pray. It's test it now. And you can test it with - you don't need to
be a big company - you can test it by buying Google AdWords
or Facebook Self Service ads. You can put these links out. You could spend 50 bucks a day and know most of what we know.
So, it's an awesome way...”
Mark Pincus – Founder & CEO @ Zynga