© 2017 TM Forum | 1 Realizing the Potential of Machine Learning Hervé Bouvier - Pauline Maury - linePaul Andreas Polz - BearingPoint
© 2017 TM Forum | 1
Realizing the Potential
of Machine Learning
Hervé Bouvier -
Pauline Maury - linePaul
Andreas Polz - BearingPoint
© 2017 TM Forum | 2
Agenda
▪ Who we are
▪ What is Machine Learning & its benefits for Telcos
▪ What is HyperCube and Why it can help
▪ Our experience – Case study
© 2017 TM Forum | 3
Agenda Who are we ?
▪ A data science consultant team of
50 people in France and 80 in
Europe with:
▪ PhDs in machine Learning, analysts
▪ Big data (Hadoop) architects &
developers
▪ Full stack Web Developers
▪ Mastering a wide range of machine
learning methods & developing A
unique & proprietary Smart
Analytics technology- HyperCube
© 2017 TM Forum | 4
What is a Datascientist ?
What his friends think he is What his mother think he is What society think he is
What his manager think he is What he think he is What he actually is
© 2017 TM Forum | 5
« Machine
Learner »
Developers
Architects
Statisticians
Business
Experts Data Analysts
Data
Science
TraditionalResearch
ComputerScience
TraditionalSoftware
Subject MatterExpertise
Math &Statistics
MachineLearning
© 2017 TM Forum | 6
Agenda
▪ Who we are
▪ What is Machine Learning & its benefits for Telcos
▪ What is HyperCube and Why it can help
▪ Our experience – Case study
© 2017 TM Forum | 7
What is Machine Learning ?
Computational learning using
algorithms to learn from and
make predictions on data
DatasetData
preparation
Model Validation Prediction
Test DataTraining data
TA
RG
ET
© 2017 TM Forum | 8
Customer
Analytics
Operational
Analytics
Fraud & Risk
Analytics
ACQUIRE | GROW | RETAIN MONITOR | DETECT | CONTROL MANAGE | MAINTAIN | MAXIMIZE
▪ Which of my clients are likely to
accept upsell offer planned in my
next DM campaign?
▪ What are the intervention rules
which would help to improve
customer satisfaction?
▪ How to develop sales performance
across my retail network?
▪ How do we identify, measure and
mitigate fraud, especially ones that
are hard-to-detect and low
frequency/high impact?
▪ Which of my clients are likely to
drop out of my loyalty program?
▪ How can we optimize costs to settle
and costs to serve in claims
handling?
▪ How to optimize my resource
allocation for preventive
maintenance effort?
▪ What are the reasons for journey
delays and levers for better
planning and accuracy?
▪ How to Support frictionless travel
across multiple modes of
transportation?
We address a wide range of business issues by
unleashing value from operational data
© 2017 TM Forum | 9
Fraud
detection
> Mitigate losses
via fraudsters
profiling
Customer
Targeting
> Boost
enrollment /
Upsell Program
efficiency
Network
Experience
> Anticipate
failures &
Cust. felt
experience
Churn
prediction
> Identify risky
cust., listen to
concerns & push
custom offersTelco Usage
Cust.
Interactions
Contracts
Sta
tic
info
rma
tio
n
Beh
avo
ria
l
info
rma
tio
n
Cust. profile
Payment
incident risk
> Profile & assist
unwealthy
customers
PoS
Performance
> Leverage best
practices to drive
perf. across network
Client Service
Efficiency
> Maximize
satisfaction &
reduce AHT/CTO
Network
PoS
Op
era
tio
ns Recommendation &
Personalization
> Build-up 360° view
& enhance
marketing
effectiveness
Machine Learning addressing TelCos stay awake issues
© 2017 TM Forum | 10
Agenda
▪ Who we are
▪ What is Machine Learning & its benefits for Telcos
▪ What is HyperCube and Why it can help
▪ Our experience – Case study
© 2017 TM Forum | 11
A cutting-edge analytics platform that derives operational insights and provides amazing accuracy
and stability in predictive modeling
What is HyperCube ?
CONNECT
DATA
VIZUALIZE
DATA
EXPLORE
DATA
MODEL
DATA
interact with your
data quickly and
intuitively
gain insights on key
drivers and related
relationships
generate predictive models
and measure performance in
a few simple steps
Connect easily to
existing sql/NoSql
database
Along with its proprietary algorithm, it provides a selection of open source state-of-the-art
algorithms and a framework to develop and deploy customized business apps tailored to
clients needs
© 2017 TM Forum | 12
Visualisation
Vac25 – Logistique
Regression 50_var
Vac25 – Gradient
Boosting 50_var
Random
Vac25 – HyperCube
– 50_var
Vac25 – Random
Forest 50_var
Vac25 – Log Reg 50_var
Vac25 – Gd Boosting
50_var
Random
Vac25 – HyperCube – 50_var
Vac25 – Rand Forest 50_var
Var25 - Logistique
Regression 50_var
Vac25 – HyperCube –
50_var
Vac25 – Gradient
Boosting 50_var
Vac25 – Random Forest 50_var
?
Prédiction
AnalysePrescription
Data
management
© 2017 TM Forum | 13
Explain & Predicty at the heart of HyperCube value proposition
Why
Who
Understand drop
out rationales
Anticipate and
target customers
Critical Business Issues Purpose
my customers
are willing to
leave?
are the most
likely to
leave?
EXPLAIN
PREDICT
Outcomes
… are 3.5x riskier
Age < 35
Owns Product A
Contract Tenure [2;5]
Analytics insights
▪ Features selection
▪ Critical threshold
▪ Business rules
▪ Data Vizualisation
Client 123
Client 232
Client 133
…
Client 211
Client 121
Scoring & Local drivers
Clie
nts
with
1
0,91
0,8
…
0,15
0,1
Billing / Age / Usage PdtA
Tech Issue / HMoving / Age
Usage Pdt B / Billing / Gender
…
Usage PdtA / Tenure / Age
Usage PdtA / Tenure / Billing
Illustration with loyalty management
© 2017 TM Forum | 14
BearingPoint helped Telco operators to successfully optimize their operations
Network preventive
maintenance
Customer satisfaction /
Inbound call prediction
Business Challenges What we did Our Cients
▪ Find out patterns in core and access network to enhance customer experience & increase cost efficiency
▪ Prepare framework to establish preventive maintenance in a continuously learning organisation
▪ Understand root causes of incoming calls from high value customers
▪ Predict customer base propensity to contact pro actively Client Service
Employee Satisfaction
▪ Analyze employee level of usage of HR Dpt service portfolio
▪ Define employee segmentation (clustering) related to HR service usage
▪ Build-up predictive models to anticipate HR needs per employee and enhance relevance of HR
push notification
Fraud▪ Profiles fraudsters and key drivers for non-payment behaviors
▪ Build-up predictive models at activation and after first 4 weeks of activity to assess level of
fraud risk
Churn prediction ▪ Build-up predictive models to anticipate level of churn risk across B2C customer database
PoS Network performance ▪ Identify key drivers for Point of Sales performance defined as tNPS, Opex intensity & Market share
▪ Build-up specific action plans for both existing store concepts
© 2017 TM Forum | 15
Agenda
▪ Who we are
▪ What is Machine Learning & its benefits for Telcos
▪ What is HyperCube and Why it can help
▪ Our experience – Case study
© 2017 TM Forum | 16
Use case 3Use case 2Use case 1
Telecom
Fraud
Reduce non-payment incidents for a Telecom Operator
Our results
Context & Challenges
➢ Actions plan &
Quick
wins identified
➢ Fraud predictive
model ready for
industrialization
Est. ROI :
300k€+/year/fraud rate point
400k+clients
7%+fraud rate
500+variables
▪ Mobile handset
subsidization at risk
due do fraud rate
level increase
▪ Need to revamp
current targeting
methods
▪ Willingness to
understand & profile
fraudsters vs good
payers
share of customers covered
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 20% 40% 60% 80% 100%
Shar
e o
f n
on
-pay
ers
cove
red
Activation model
Activation +Activity model
Client currentscoring
Wizard
© 2017 TM Forum | 17
Fraud
Revenue
Contract
Cust. Profile
▪ Age
▪ Gender
▪ Localization
▪ Correspondance
vs billing
▪ COGS
▪ Price
▪ Offer type
▪ Subsidy level
▪ Billing cycle
▪ Total revenues
▪ Revenues from roaming
▪ Revenues from data
Usage
▪ Onnet/offnet calls
▪ Ratio calls in/out
▪ Sms/mms in/out
▪ Data volume
Nb of inbound calls
Nb sms sent
Customer Revenue
Nb of outbound calls
MFU Handset
Duration data usage
Variable Set Variables ranking
▪ Activation
Channel
▪ Activation date
▪ Salesman code
▪ Agent
Context Device
▪ MFU handset vs
subsidized
▪ Smartphone Y/N
▪ Model brand
▪ Model price
sta
tic
be
ha
vio
ral
Use case 3Use case 2Use case 1
Telecom
Fraud
Reduce non-payment incidents for a Telecom Operator
© 2017 TM Forum | 18
HyperCube helped to find out influencial factors and specific local profiles
standing for a high level of risk
Customers that match the following conditions …
This rules concerns:
5% of Fraudsters
2% of total customers
… are 9 times more likely to stay loyal
Activation Channel TELESALES
OUTBOUND
2,6
between 17
and 40
5,1
9
ABC
Is
Is
Is
Offer
Age
0
50000
100000
150000
200000
250000
300000
0
0.5
1
1.5
2
2.5
3
Customers recruited via Telesales outbound are 2.6 times more
likely to be non-payers
Use case 3Use case 2Use case 1
Telecom
Fraud
Reduce non-payment incidents for a Telecom Operator
© 2017 TM Forum | 19
01,0002,0003,0004,0005,000
Alarm_1
Alarm_2
Alarm_3
4300
2500
800 1000
0
2,000
4,000
6,000
Boardtype 1
Boardtype 2
Boardtype 3
Boardtype 4
# alamrs
Use case 3Use case 2Use case 1
Telecom
Preventive
maintenanceAlarms’ preventive maintenance: Increase customer experience
Context & Challenges➢ Short term solution: Implement an Early Warning Dashboard
(E.W.D. - Operations cockpit with daily reports)
Improve response time and avoid “blind spot” outages:
preventive maintenance
➢ Long term solution: the data structure and quality was not
sufficient for most analyzed data sources
Establish a structured and comprehensive data warehouse
(DWH)
Introduce Data Mining technologies and methodologies to
improve the data quality and enable detailed analytics
Raise network quality
• Higher network stability
• Upfront identification of incident root causes
• Faster reaction to incidents
Increase cost efficiency
• Reduction of costs of operations
Enhance customer experience
• Enhance customer communication
by better knowing the network and
network event
© 2017 TM Forum | 20
Use case 3Use case 2Use case 1
Telecom
Preventive
maintenance
We analyzed different alarm types from January to May according to their
impact on the network
Alarms’ preventive maintenance: Increase customer experience
N0= 43,634,389
Filter for German alarms
N1= 2,912,658
Filter for relevant alarms
N2 = 1,761,884
ALARM_1 ALARM_2 ALARM_3 ALARM_4 ALARM_5 ALARM_6 ALARM_7 ALARM_8 ALARM_9
Alarm dataInventory
dataDevice data
Geographic
al data
© 2017 TM Forum | 21
Use case 3Use case 2Use case 1
Telecom
Preventive
maintenance
We identified regional alarm concentrations and local problem spots which
covered the majority of alarms in the areas in different cities in Germany
Alarms’ preventive maintenance: Increase customer experience
• Regional alarm concentrations and rules were identified where under specific hardware and software setups alarm concentrations occurred
• Local spots in large and small cities could be due to single incidents or single problem boards which caused a majority of the identified alarms
• The geographical location and the socio-economic factors did not have a significant influence on the number of alarms on the network elements
*Anonymised data
Berlin* Pattern – Rule on the average number of alarms
Under the following conditions …
✓ Board Type is Board A
✓ Hardware Version is B
✓ Software NE Version is NEv2.02
✓ Software Board Version is v1.01
✓ Ort is Berlin*
This rules concerns:
▪ 87,2% of all occuring alarms in Berlin*
▪ 30,3% of the total number of alarms matching:
➢ « Board type »
➢ « Hardware » and
➢ « Software »
The average number of alarms per board is 36,2
times higher than the average of all locations!
© 2017 TM Forum | 22
Use case 3Use case 2Use case 1
Telecom
CRMUnderstand & Predict Customer service inbound calls
Targeting enhanced generating short term
business impacts
Ex : ~500k€ contact cost savings / mktg campaign > Telecom
Robustness over the time that limit models
updating effort
Ex : <4% loss of prediction accuracy after 3 months > Telecom
Potential synergies with others tools & methods
Ex : Up to 40% of additional targets list with standard tools
Build classifier to predict future Client
Service caller
1
Compare qualitatively analysis outcomes to already
existing analysis performed by marketing teams
▪ Ability to map and confirm proven facts & figures
▪ Capacity to increase current understanding with new
insights
Determine root cause of Client
Services inbound calls
2
© 2017 TM Forum | 23
Use case 3Use case 2Use case 1
Telecom
CRM
HyperCube has ingested and analyzed a large volume of information to
ensure results completeness and accuracy
Understand & Predict Customer service inbound calls
Client
Age
Contacts
Appels en CC
# campagnes MD reçues
#interventions tech terrain
# visites magasins
# total interactions client (+hist.)
selfcare
# sms sortants
Revenus
ARPU
Evolution ARPU
#incidents impayés
# & durée
suspensions abo
Contrats
Ancienneté Orange
Options activées
Actuelle
Ligne de marché
Précedente
Usage
VoixData
TV
Surappels
Sexe
Localisation
CSPAncienneté offre
Offre
Pay. Grat.
# migrations
Nature Dates
Pay. Grat.Actuelle Moy. Hist.
…> fixe > mobile
browsing sms mms
Internet…
voix
data
Roaming
Alertes
service
roaming …
# visites #modifs contrat# clicks
M-xN-x
…
volume motifsM1
M2…
Forf.Hors forf.
Internet
#déménagements
1MCustomers
10 kVariables
56 kCallers
1,7Call/caller
© 2017 TM Forum | 24
Use case 3Use case 2Use case 1
Telecom
CRM
Among full set of customer data, few are correlated to significant rate
of Client Service inbound calls
Understand & Predict Customer service inbound calls
22.520.017.515.012.5 25.0 32.530.027.5 55.010.0
38
36
34
26
24
22
20
18
16
14
12
10
8
6
4
2
0
37.535.0
Shar
e o
f ca
llers
Acq/Ter insurance option Feb
Unlocking FebMobile Change Program Feb
Acq/Ter paying option Feb
Migration Feb
Handset renewal Feb
Dunning Process Feb
Caller rate
Gesture of Goodwill MarAt least 1 connection to coordinates webpage Feb
Extra Call Pack > 11 euros Jan
Extra Call Pack > 88 euros Jan
Segment Value 5+ Feb
At least 2 bills unpaied Feb
At least 1 bill unpaied Feb
OS Change Feb
Claims
Billing
Segment
Payments
Technical after-sales
Sales
Dunning
Selfcare
© 2017 TM Forum | 25
Use case 3Use case 2Use case 1
Telecom
CRM
Migration is the most significant reason for contacting Customer Service but
combined with handset renewal, value segment or invoice issues.
Understand & Predict Customer service inbound calls
Customers matching the following conditions:
Are 15x more likely to contact CS
Union of those customers stands for 7% of incoming managed by CS (i.e. 76k calls) > very low segments intersection
Handset renewal effect
Customers willing to change their mobile Offer are 5,2x more likely to get in touch with Customer Service
Are 9x more likely to contact CS
✓ Has migrated
✓ Belongs to 5+ value segment
✓ Has already had a non-paying
incident
✓ Has migrated
✓ Has renewed handset
Are 9x more likely to contact CS
Value segment & non-paying
incident historic effects
✓ Has migrated
✓ Has consulted recently online
billing
✓ Has reduced invoice amount by
> 11€ (1month later)
Invoice issue effect