Business model innovation by experimentation

Business Model Innovation by ExperimentationYoav AviramEnergized Work@yobo

IT Projects60% don’t meet schedule, budget or quality goals - IBM

50% failed to achieve what they set out to achieve - KPMG

17% go so badly that they can threaten the very existence of the company. - Calleam Consulting

Startups 95% of technology startups fail - Allmand Law

93% of the Angel investments never achieve expected ROI - University of Washington

80% of the VC investments never achieve expected ROI - National VC Association

WHY?

Software: Tip of the Iceberg

Too late

Poor quality

Missing functionality

Agile

An iterative and incremental approach to development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams

- Wikipedia

Who is the customer?How do we acquire new users?How do we convert users to paying customers?How should the product look and behave?How is our product used?What problems does the product attempt to solve?What features provide the greatest value?

...the Product Iceberg

Lean Startup

A combination of business-hypothesis-driven experimentation, iterative product releases, and "validated learning"

- The Lean Startup, Eric Ries

...the Business Model IcebergWho are our customers?Who is the competition?How do we reach our customers?What are the revenue streams?What is the investment needed?What is the operational cost?What is our unique selling point?What team members to recruit?What are our business goals?

Business Model Innovation

- Business Model Generation, Alexander Osterwalder

Product-Market Fit

There is a poorly met need a reachable market faces

There is a solution people are happy to pay for

There is a way to package and deliver the solution in a cost effective manner

This is all About Risk

Uncertainty

The lack of complete certainty, that is, the existence of more than one possibility. The "true" outcome/state/result/value is not known.

- How to Measure Anything, Douglas Hubbard

Quantifying UncertaintyA set of probabilities assigned to a set of possibilities

“There is a 60% chance this market will double in five years”

Risk

A state of uncertainty where some of the possibilities involve a loss, catastrophe, or other undesirable outcome.

- How to Measure Anything, Douglas Hubbard

Quantifying Risk

A set of possibilities each with quantified probabilities and quantified losses

“There is a 40% chance the proposed oil well will be dry, with a loss of $12 million in exploratory drilling costs”

The Problem with Risk & TechnologyTechnology is used to solve complex problems

Our mind is wired to oversimplify complexity (we are not very good at understanding probabilities)

We are solution oriented

We are optimistic (overconfident) by nature

People (grossly) underestimate risk & uncertainty

Compound RiskCombination of two or more related risks

Compound probabilities are (very) counter intuitive

Monty Hall

Contestants on a game show are given the choice of three doors.

Behind one door is a car, behind the others, goats.

After a contestant picks a door, the host, who knows what's behind all the doors, opens an unchosen door, which reveals a goat.

He then asks the contestant, "Do you want to switch doors?"

Should the contestant switch doors?

Yes!The odds of winning are 2 out of 3 if you

switch

Compound Probabilities

This answer is so counter intuitive, most people get it wrong

It’s why they made a game show off of it: the trick works every time

The lesson:

We need to give serious consideration to risk

We must not rely on our intuition when factoring risk & uncertainty

We need a better accounting system

Risk Management for TechiesAvoid by eliminating the situation or activity that presents it

Transfer through insurance or through other types of contracts

Reduce by hedging your bets or reducing uncertainty

Retain, because some risks are worth assuming

Risk Reduction

Investor Mindset

Spread your bets on a portfolio of investments instead of one

Decide on an investment strategy

Split the investment into small incremental bets: abort bad products quickly and ramp up investment in the good ones

Uncertainty Reduction

Value of InformationInformation reduces uncertainty

Reduced uncertainty improves decisions

Improved decisions have observable consequences with measurable value

- Information Theory (1948)

Sources of InformationExisting within the company

Other’s research

Experiments

Live product

A scientific procedure undertaken to make a discovery, test a hypothesis, or demonstrate a known fact

- Wikipedia

Experiment

Experimental Discovery

Prioritising ExperimentsUncertainty around making a decision is high (little existing information)

The value of the opportunity, or the cost of a mistake, is high

Experimentation is least expensive in terms of cost and time

Designing Good Experiments

Attempt to falsify a hypothesis

Are objective, measurable and repeatable

Are controlled (variables tested in isolation)

Are cost effective

Influence a decision

Experiments aren't free

It's an investment decision

Information may be cheaper elsewhere

Measurement

A measurement is an observation that results in information (reduction of uncertainty) about a quantity

Qualitative Measurements

The soft stuff

Descriptive, subjective, messy and hard to quantify

Customer surveys, focus groups, interviews

But … provide insights about Perceived Value

Good Qualitative QuestionsAvoid leading phrases such as "do you agree that..." Make customers part with money or sign up, rather than asking them whether they wouldUse follow up questions to get to the ‘why?’Qualitative data can easily be influence by cognitive biases

Quantitative Measurements

Start by decide on the right Metric (what to measure)

Then set lines in the sand: what success and failure look like

Always validate results with qualitative data

Good MetricInfluences a decision

Comparative

Rates and Ratios are easier to compare and act upon

Measure what’s important to your customers

Putting it all TogetherInvest wisely, through small incremental bets to reduce risk and discover opportunities

Information reduces uncertainty & risk

Experiments are not the only source of information

Experiments have the biggest impact in a high value, high uncertainty and little data situations

Disprove hypotheses, don’t confirm them

Think in terms of Risk and account for it

The distinction between business and technology is anachronistic

Reading List

The Lean Startup by Eric Ries

Business Model Generation by Alexander Osterwalder

How To Measure Anything by Douglas W. Hubbard

Lean Analytics by Alistair Croll and Benjamin Yoskovitz

Questions?

Yoav AviramEnergized Work

yoav@energizedwork.com@yobo

Accounting for Risk

Frame your business and product as a set of hypotheses

Declare your assumptions and how you can prove them wrong (falsifiable)

Evaluate your results ruthlessly, and be prepared to change course

Assumption BacklogAssumption Certainty Risk ($) Critical

People like shiny apps 90% $100K (cost of beta) Yes

Market size for shiny apps at least 200 million people

50% $1M (cost of a full product) Yes

I can build a shiny app 95% $50K (cost of development) Yes

People share shiny apps with friends 45% $100K (cost of beta) No

5% of users will upgrade to shiny+ for $5 a month

20% $1M (cost of a full product) Yes

10% of active users will spend $2 a month on shiny accessories

75% $100K (cost of beta) No