Release Cohorts - Mobile Marketing Meetup Feb 2013

Know the impact of every releaseAlign product development with engagement metrics

storyberg.com

Lean Product Development

If you don’t measure properly

then you don’t learn

“If we have data, let’s look at data.

If all we have are opinions, let’s go with mine.”

– Jim Barksdale, former Netscape CEO

“I’d encourage everyone doing a web startup to adopt

the startup metrics methodology and within that methodology,

make sure you are looking at cohorts of users,

not just all of your users in the aggregate.”

– Fred Wilson

Cohorts are …

a group of people who share a common experience

Current Cohort Solutions

Experience is that we signed up together

Release Cohort

Users experience based on the

feature set — driven by releases

Developers release features to …

(1) Activate new users

(2) Continue to engage users

(3) Re-activate old users

Compare how releases impact users

over time to make sure you’re building

something people want

Funnel Analysis

Funnel analysis tries to predict user behavior

Feature Cohorts

Learn what users do after seeing a new feature

Feature Validation

New Feature: Experiments

In order to know I’m building something people want

As a developer

I want to validate my features

Key Activities (to track)

Install JS Identify Users Track Events (key activity) Paid

Experiments Cohort

Learn what users do after seeing the

new feature

Drive Product Changes

Learning: Creating an experiment increased key activity

Action: Move create experiment to welcome flow

Javascript API

Auto-detect new features<div data-sb-tag=”New Welcome Flow”>

<!-- CODE -->

</div>

Record event_sbq.push(['event', {

name: 'set status',

user_id: '1',

}]);

Identify Users_sbq.push(['identify', {

user_id: '1234',

email: '[email protected]',

sign_up_date: 1330779887,

name: 'Test User'

}]);

Ruby Gem

Record eventStoryberg.event(

'activity',

current_user.id

)

Identify UsersStoryberg.identify(

current_user.id,

{

email: current_user.email,

name: current_user.full_name,

sign_up_date: current_user.created_at.to_i,

sb_tag: ‘New feature’

}

)

Storyberg next steps …

(1) Provide automatic insights

(2) Improve Cohort Analysis so you know what your best users do

(3) Improve A/B testing workflow (everyone should be doing it)

Discussion

What would a mobile SDK look like?

Michael [email protected]

storyberg.com