P 01 paw_methods_2017_10_30_v4

A METHOD TO AI MADNESS

Vishwa Kolla Head, Advanced Analytics

John Hancock Insurance

TOPICS

Background Framework Case Studies

2

BUILD ME A MODEL TO …

3

REDUCE COMPLAINTS

GROW WALLET-

SHARE

GROW CSAT

REDUCE CHURN

REDUCE COST TO

TARGET

GROW BOTTOM-LINE

GROW TOP-LINE

REDUCE COST TO

ACQUIRE

MODEL BUILD IS THE PATH OF LEAST RESISTANCE

4

Platforms

R

P H20

SPARK

TENSOR

FLOW

TBD

SUPERVISED

UN

SUPERVISED

NLTK NUM

PY

PAN

DAS

PLOT

LY

PLA

TFO

RM

S

ALG

OR

ITHM

S

PA

CK

AG

ES

PYO

DBC

CRY

PTO

PYPD

F

SCI

KIT

TOR

NAD

O

ZICT BAB

EL

BLA

ZE

A THOUGHTFUL APPROACH CAN YIELD BETTER OUTCOMES

BUSINESS

USE CASES DATA MATH

TECHNICAL

IMPL.

BUSINESS

IMPL.

FEED

BACK

5

TOPICS

Background Framework Case Studies

6

“A” MODEL BUILD FRAMEWORK

7

DATA TARGET CONSTRUCTION EVALUATION PERFORMANCE

SOURCES

DISTANCE FROM

SIGNAL

SAMPLING

METHOD

SAMPLE

SIZE

SIGNAL SIZE

PREDICTION

HORIZON

UNIT OF

ANALYSIS

ONE MODEL vs.

STRATIFIED

ONE MODEL vs.

SEVERAL MODELS

TARGET

DEFINITION

PRESENCE OR

ABSENCE

BLACK BOX vs.

CLEAR BOX

RECENCY

FREQUENCY

SEVERITY

FEATURE

SELECTION

MODELING

STRATEGY

MODEL

STRENGTH

EXPLANATORY vs.

IMPORTANCE

ACCURACY vs.

SENSITIVITY vs.

SPECIFICITY

ECONOMIES OF

SCOPE

MODEL

FIT

BAGGING

ENSEMBLE

SINGLE vs.

MULTIPLE STAGES

PREDICTION &

OPTIMIZATION

BOOSTING

TOPICS

Background Framework Case Studies

8

Business

9

IT (ALWAYS) STARTS WITH A BUSINESS PROBLEM

PROSPECTING NURTURE ACQUISITION

MARKET

SEGMENTS

CUSTOMER

SEGMENTS

LIKELY TO [*]

MEDIA

MIX

CHANNEL

SURVEY

ANALYTICS

CROSS / UP-

SELL

OCR

MISREP

LIKELIHOOD

MORTALITY

APS

SUMMARY

FLUIDLESS

SMOKER

LIKELIHOOD

MORBIDITY

CHURN

NEXT BEST

OFFER

CLAIM

LIKELI-

HOOD

JOURNEY

CLAIM

SEVERITY

NEXT BEST

ACTION

FRAUD

>>

TEXT

ANALYTICS

OPTIMIZE

NEXT LIKELY

ACTION

WELLNESS

IOT

ANALYTICS

NPS

ANOMALY

>>

10

FOCUS ON INCREMENTAL VALUE KEPT US GROUNDED

BUSINESS CASE OPTICAL REALIZABLE SHARED INCREMENTAL

IN PROSPECTING, TARGET OPTIMIZAITON IS A JOURNEY

12

… LOWER CUSTOMER TARGETING COSTS A SERIES OF OPTIMIZATION TARGETS …

Prospects

Leads

Apps

Issued

Placed

CPL

CPA

CPP

CP[*] CHANNEL

MIX

Data

13

PLANS ARE NOTHING ; PLANNING IS EVERYTHING

14

EDA USEABLE

USEFUL

DERIVATIVES

BI-VARIATE CROSSTAB PRINCOMP JOURNEY

A DATA STITCH IN TIME SAVES NINE

15

CLAIM TERMIN

ATION

CLAIM

ACTV. DEMOS

CALLS

INTERA

CTION

CLAIM

INIT.

CUSTOMER MONTH

FRAUD DETECTION

UNDERSTANDING DATA SAVES (NOT WASTES) TIME

16

Signal

Distribution

Pop. Incidence Rate

Skews Model Inclusion

Math

17

FLEXIBILITY IN TARGET DEFINITION IMPROVED ACTIONABILITY

18

2017 2014

Predict incidence

In next 3 years

2007

2017 2014

Predict incidence

3 years out

2007

Vs.

RIGHT SIZING SIGNAL CAN YIELD BETTER OUTCOMES

19

SIGNAL

DILUTION

SIGNAL

AMPLIFICATION

1%

99%

40%

60%

SIMPLE MODELS CAN HELP US EXPLAIN BIG DRIVERS

20

PREDICTORS

PRESENCE

RECENT

FREQUENT

SEVERE

QUANTIFICATION OF INFORMATION GAP IS A GOOD FIRST STEP

© Andrew Ng

INFORMATION GAP

WINNING

MODEL

CHALLENGING CHAMPIONS HELPS US UP THE ANTE

22

DATA TARGET

SOURCES

DISTANCE FROM

SIGNAL

SAMPLING

METHOD

SAMPLE

SIZE

SIGNAL SIZE

PREDICTION

HORIZON

UNIT OF

ANALYSIS

ONE MODEL vs.

STRATIFIED

ONE MODEL vs.

SEVERAL MODELS

TARGET

DEFINITION

METHODS

LINEAR

TREES

DEEP-

LEARNING

EVALUATION

MODEL

STRENGTH

EXPLANATORY vs.

IMPORTANCE

ACCURACY vs.

SENSITIVITY vs.

SPECIFICITY

ECONOMIES OF

SCOPE

MODEL

FIT

Technical Implementation

23

MULTI-STAGED MODELS PROVIDED IMPLEMENTATION FLEXIBILITY

24

9-1

0

1-8

1-7 8-10

Likely to

Qualify

Likely to

Respond

Sweet

Spot

DESIRED

SIGNAL

MODEL

MIS-

CLASSIFICATION

MODEL

EXCLUDE NOISE 1

INCLUDE MIS-CLASSIFIERS 2

STAGES

Business Implementation

25

A CULTURE OF MEASUREMENT, TEST AND LEARN IMPROVES VALUE

26

Measure

Test Learn

Build

CONTINUOUS

IMPROVEMENT