Dec 21, 2015
Charles Schwab: Optimized Customer Experiences with Big Data and Oracle Real-Time Decisions [CON5352] Customer Experience Optimization
Andy Welch, Principal Architect Charles SchwabRich Masi, Partner NewVantage PartnersJoe Khazen, Director, Real Time DecisionsOctober 1, 2014
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
3
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 4
Agenda
1
2
3
Overview of Real Time Decisions
Charles Schwab-RTD and BigData Use Case
Future Plans
Q & A4
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |5
CREATING GREAT EXPERIENCES IS THE IMPERATIVE
ATM
PORTAL
BRANCH
Call Center
WEB
MOBILE
SOCIAL
KIOSKS
“know me, and wow me”“understand me, and reward me”
“meet me, and engage me”“delight me, and guide me”
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |6
LEVERAGEDATA
OPTIMIZEEXPERIENCE
ADAPT QUICKLY
What We Hear from Customers
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |7
SocialIn Store Contact Center
Field Service
Channel SalesCX for
SalesCX for
MarketingCX for
Commerce
Real Time Decisions (RTD)
CX forServiceCloud
PlatformServices
SocialPlatformServices
Common Hardware Systems Infrastructure
Direct Sales
Web
Mobile
Oracle’s Customer Experience Portfolio
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |8
Optimize the Customer Experience with RTD• Flexible Way to Make Decisions
– Single decision engine supporting a consistent customer experience across all channels
– Easily Integrated into Existing Applications– Goals, Rules, Models, Optimization, Arbitration
• Automated Self Learning– Incrementally builds Analytical models for Learning
and Decisions– Analytical Adaptive Decisions
• Quantifiable Results– Quantifiable and Measureable Lift on Each Project
– Various Test and Control Scenarios
Optimal & Personalized Customer AND Business Centric
Recommendations
KPIArbitration
Eligibility Rules& Models
Offers/NBAs
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Real time contextual data
Historical data
Relevant external sources eg Social Media
Target Audience
Predictive Modeling
+
+
+ =
σ
Exceptional Customer Experience
Self-LearningLoop
Personalized recommendations,
offers & actions
Real Time Decisions
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal/Restricted/Highly Restricted
10
Oracle Customer ExperienceOptimize Every Decision
Marketing• Customer experience
optimization– Themes, Colors, Navigation
• Next best offer
• A/B and multi-variant testing
• Content personalization
Service• Service treatments
optimization
• Customer retention programs
• Call center optimization
• Risk and fraud analysis enhancement
• Next best action
Sales• Customer Acquisition
• Cross Sell/Upsell
• eMail Personalization
• Offer Optimization
• Next best action
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |11
12/20/2012 11
11
Optimized Customer Experiences with Big Data and Oracle Real-Time Decisions
Andy WelchPrincipal Architect, Charles Schwab
Rich Masi,Partner, NewVantage Partners
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |12
12/20/2012 12
Why Oracle Real-time Decisions
• Speed• Context sensitivity• Scalability
Technology Foundation• Marketing velocity• Goal management• Analytical ideation
• Across a channel experience• Across channels• Across a customer lifecycle
Scope of Decisions Marketing Operations
World-Class Decision Management
Key Functional Architecture Component Areas.
1. Open architecture to support real-time communication with channels2. Decision management business interface to manage a decision set and decision strategy3. Decision service that returns an optimized result to channel requests in real-time4. Learning service that is updated in real-time based upon channel actions5. Open architecture to support integration with data systems including a big data platform6. Business insight platform for reporting and analytics
Goal: Optimize customer experience by delivering relevant contentOracle RTD was selected because it best met functional needs for a world-class real-time decision
management environment
12
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |13
12/20/2012 13
13
RTD at Charles Schwab
• Initial use of RTD to support content in real-estate across hundreds of Schwab.com pages
• Supports millions of optimized content requests per day
• Meets very tight response time SLA
• More than double the response rate versus legacy approach
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |14
12/20/2012 14
Learning Service
Decision Service
14
RTD Database
Big Data Cluster
1. login sends start session informant to RTD with a customer identifier when the user has been successfully authenticated
2. RTD sources the customer profile from the big data cluster using the customer identifier though a custom integration and establishes RTD session
3. RTD sources choice history from RTD database4. Page visit sends Advisor request for optimized content to RTD.5. RTD decision service processes business rules, predictive analytics
and decisions6. RTD returns an ordered list of content ids to page7. Web page calls content system to render content8. A content response sends a response informant to RTD9. RTD updates session profile10. RTD logs choice history and learnings11. Learning service processes learning records
1
5
RTD
RTD High Level Flow
4
3
RTD Session2
6
CMS7
8
9
10
11
Login Pages
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |15
12/20/2012 15
15
Enterprise Decision Management
• Created a decision management taxonomy that maps to business stakeholders management practices.
• Designed this structure to support:
Enterprise level goal management practices Multivariate testing at a content level Multiple channels Many placements (pages) Many slots (page positions) per placement with varying number of items returned per slot Many slot types (content types) for placements Decision strategy testing within and across placements and slots A learning graph based upon content, user and placement metadata dimensions
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |16
12/20/2012 16
16
Initial Test Design
BAU Random Group
Random 5% of
population
Random selection of
BAU banners
Business Rules Group
Random 15% of
population
Random selection
within business
rules
Rules + Analytics
GroupRandom 80%
of population
Response likelihood selection
within business
rules
• Visitors are placed in a group in real-time based upon random assignment which is persistent for future visits.
• Response rate for Business Rules Group is significantly greater than the Business as Usual (BAU) Random Group
• Response rate for Rules + Analytics Group is more than double the Business as Usual (BAU) Random Group response rate
• Group membership and real-time likelihood score are written to table with each event
• Business can add other groups or change definition of current groups through a business interface
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |17
12/20/2012 17
17
Decision Processing
Calculate likelihood of
response
Business Rules Group
Filter for content eligibility
Assign random score
Determine Group Membership
Eligibility Rules
Score Content
Process Decision Strategy
Arbitrate based upon likelihood
score
Filter for content eligibility
Rules and Analytics
Group
Arbitrate based upon random
score
BAU Random Group
No eligibility
rules
Assign random score
Arbitrate based upon random
score
Group membership,
rules, and decision
strategy are all configurable
through a business interface
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |18
12/20/2012 18
Multivariate Testing
130.000%
0.001%
0.010%
10,000 100,000 1,000,000 10,000,000
Presented Count by Response Rate
Response Rate
Presented Count
Quadrant #1 Choices
presented often and
performing well
Quadrant #2 Choices
presented less often and
performing well
Quadrant #3 Choices
presented less often and
performing not as well
Quadrant #4 Choices
presented often and
performing not as well
Best Practices Benefits Bucket
Define a cycle time period (week) for test and learn activities
Build Best Practices
Beginning of period, monitor the
program changes and identify opportunities
Grow Best
Practices
Middle of the period, review opportunities
with business and make change
decisions
End of the period, start
change process and make
changes that can be made.
• Rolling out multivariate testing practices to promote continuous campaign improvement
• Based upon a proven practice developed over 15+ years of real-time marketing program optimization
• Practice identifies program areas with the most opportunity for improvement and aligns marketing levers with improvement areas
• As a continuous champion-challenger optimization strategy, the value received from these practices compound rapidly and are typically are the largest value driver for programs
18
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |19
12/20/2012 19
19
Learning Graph
Placement
Slot
Slot Type
Channel
User OS
User Application
User Device Type
General Subject
Detailed Subject
Category
Content
• Created a learning graph by configuring RTD to learn on metadata of content, placement, and user info
• All presentation and response events are written to predictive analytics that build a response profile of higher order elements up and across the learning graph
• All analytics partitioned by channel to allow for multi-channel rollout
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |20
12/20/2012 20
20
RTD / Big Data Integration
RTD Database
RTD
RTD Session
Data Service
Decision Service
Learning Service
Java GetProfile
Data Cluster
• Developed custom big data integration with RTD.
• Big Data Cluster has a robust customer profile information for millions of customers.
• Creates customer profiles from multiple data sources and application-specific definitions.
• Significantly outperforms benchmark database retrieval
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |21
12/20/2012 21
21
Highly Available and Scalable Environment
• Highly Available: Multiple Active/Active Data Centers
• Scalable: Multiple servers running Decision Services in each location with learning service on separate instance
DR DB Replica
Web Page
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |22
12/20/2012 22
Next Steps
Decision strategy testing Multivariate testing Rollout to additional placements/slots Enhance data model Expand channels
22