Technical Master Class
17 July 2014
Accelerate Digital London
• The Challenge
• Shortcomings of the current Enterprise IT
• Introduction of the "Digital Business Platform"
• Location of the Digital Business Platform in the Enterprise Architecture
• Digital Business Platform and Orchestration
© 2014 Apigee Confidential – All Rights Reserved 4
Partner Integration
Web site
Wearable Electronics
Gaming Consoles
Smart Appliances
Smart Sensors
Smart phones
Connected Cars
Tablets
KiosksAPI
API
Your Digital Assets
AP
I AP
I
Your Digital Assets
The changing demands for Enterprise IT
• The Challenge
• Shortcomings of the current Enterprise IT
• Introduction of the "Digital Business Platform"
• Location of the Digital Business Platform in the Enterprise Architecture
• Digital Business Platform and Orchestration
© 2014 Apigee Confidential – All Rights Reserved
SOA implementation projects
• choose tools with care
• built well designed services
• designed contracts through solid engineering principles
Overall system-to-system communication across the company got
• decoupled to ease maintenance
• properly secured
• made reliable
One of the current standards of Enterprise IT: SOA
SoRData base
Back end
SOA Middleware
SOA Service
SOA Service
The created infrastructure is great for complex business logic and process integration e.g. with long running transactions and human interaction
© 2014 Apigee Confidential – All Rights Reserved 7
Is your architecture fit for these challenges?
Partner Integration
Web site
Wearable Electronics
Gaming Consoles
Smart Appliances
Smart Sensors
Smart phones
Connected Cars
Tablets
KiosksAPI
API
Your Digital Assets
AP
I AP
ISoR
Data base
Back end
SOA Service
SOA Service
?
??
??
??
??
?
SOA Middleware
© 2013 Apigee. Confidential – All Rights Reserved© 2013 Apigee. Confidential – All Rights Reserved
Different Focus
8
DeveloperUser API API Team BackendApp
Inside-Out approach
Integration
Transactional
Predictable usage
Outside-In approach
Consumption
Interactive
Unpredictable usage
External Internal
© 2014 Apigee Confidential – All Rights Reserved 9
Aspect SOA / ESB Apps and APIs
Some Technical Aspects
Enable developers, either external or internal, to build nifty, compelling apps, and allow users to run them
High-latency, untrusted (Mobile wireless network)
Web technologies, REST, JSON, Javascript
Internet standards including oAuth and OpenID
Of primary importance
Enable Internal developers and systems to connect, while complying with IT department standards
Low-latency, trusted
SOAP, XML, proprietary
Internal mechanisms, WS-Security, LDAP
Limited use, secondary importance.
Core goal
Network
Technologies
Authentication and Authorization
Usage Analytics
• The Challenge
• Shortcomings of the current Enterprise IT
• Introduction of the "Digital Business Platform"
• Location of the Digital Business Platform in the Enterprise Architecture
• Digital Business Platform and Orchestration
© 2013 Apigee. Confidential – All Rights Reserved© 2013 Apigee. Confidential – All Rights Reserved
Backend
Speed up while preserving the existing invests
Digital Business Platform layer
SOA Middleware layer
System of RecordYe
ars
Mon
ths
IT Applications Partner Apps Mobile Clients "Internet of Things"
Day
s
Database
© 2014 Apigee Confidential – All Rights Reserved 12
• Fast, light and stateless
• Quickly configurable, but at the same time extensible
• Improve response time (caching, paging, …)
• Self service onboarding for developers
• Comprehensive and holistic view on API usage and performance
• Support both vertical and horizontal scaling
• Deployable across regions, e.g. disaster recovery, geo load balancing
• …
Selected features of a Digital Business Platform
© 2014 Apigee Confidential – All Rights Reserved
A Digital Business Platform – Logical View
Mobile clients / AppsProvide data or need access to digital assets
ESB/MQ
System of Record
System ofRecord
API RouterCalls / data needs to be routed to the ideal target
API ProxySome calls / data will be routed to an SOR
App Services Custom data for the apps
App metricsApps require metrics
API metricsAPIs require metrics
Data mining
Machine
Learning
Predictive
Analytics
Data Lake
InsightsAPIs driven by data
Developer PortalDevs need access
© 2014 Apigee Confidential – All Rights Reserved
• To achieve fault tolerance, horizontal scalability and high performance (high throughput, low latency) a Digital Business Platform needs to be thoroughly designed
Design of a Digital Business Platform
API Router
API Proxy
Data Platform
API Router
API Proxy
API Router
API Proxy
runtime db
runtime db
…
…
load balancer
Metrics
Insights
Stateless components
to easily scale horizontally
Replicating NoSQL to be fault tolerant and to scale horizontally
(even across regions)
Geo location aware, round robin,monitoring API Router/Proxy availability
Scalable, Real time access,
central data platform formodeling
Dev Mgmt
• The Challenge
• Shortcomings of the current Enterprise IT
• Introduction of the "Digital Business Platform"
• Location of the Digital Business Platform in the Enterprise Architecture
• Digital Business Platform and Orchestration
© 2014 Apigee Confidential – All Rights Reserved
A Generic Digital Business Platform – Summary
API Router
API Proxy
App Services
App metrics
API metrics
Developer Portal
Insights
Developer Management
API Services
API Intelligence
© 2014 Apigee Confidential – All Rights Reserved
The simplified new Integration Architecture
SOA Services
ExternalClients
Partner Clients
SOA Middle-ware (ESB)
Back end
API Services
Back end
Back end
Adapter
DeveloperManagement
APIIntelligence
SOAManage-
ment
DigitalBusinessPlatform
Integration / SoaArchitecture
API API
Internal Clients
Data Plat-form
© 2014 Apigee Confidential – All Rights Reserved
OUTSIDE
DMZ
It can can be deployed on premise …
ExternalClients
Partner Clients
Internal Clients
DeveloperManagement
APIIntelligence
API Services
SOA Services
SOA Middle-ware (ESB)
Back end
Back end
Back end
Adapter
SOAManage-
ment
API API
Data Plat-form
CORE
© 2014 Apigee Confidential – All Rights Reserved
DMZ
… but for agility reasons cloud is best
SOA Services
SOA Middle-ware (ESB)
Back end
Back end
Back end
Adapter
DeveloperManagement
APIIntelligence
SOAManage-
ment
API API
ExternalClients
Internal Clients
API Services
Partner ClientsOUTSIDE
CORE
Data Plat-form
• The Challenge
• Shortcomings of the current Enterprise IT
• Introduction of the "Digital Business Platform"
• Location of the Digital Business Platform in the Enterprise Architecture
• Digital Business Platform and Orchestration
© 2013 Apigee. Confidential – All Rights Reserved© 2013 Apigee. Confidential – All Rights Reserved
A composite service combines different services to deliver a value-added new service and can be categorized into different types:
1. User Experience centric Mashup
• Enhance UI
– by adding public cloud data (e.g. maps, stock info, weather info, ...)
– by combination of content from different sources (e.g. RSS, ATOM,…)
2. Intermediation / Customization
• Enhance services
– with public cloud data (e.g. maps, stock info, weather, ...) or partner data
– by payload/security mediation to homogenize the service use
3. Arbitration/Aggregation
• Unify services by combination in a read-only way or with data changes that are completely independent
• Route services to provide one access point onto several different services without knowing implementation details
4. Process Integration/Package
• Create new services by tight integration into business workflows
• Resource changes will most probably be transactional and depend on each other
Definitions
21
Unsafe with depen-dencies
Indepen-dently
consistent
Safe
Safe
© 2013 Apigee. Confidential – All Rights Reserved© 2013 Apigee. Confidential – All Rights Reserved
Placement in the Integration Architecture
22
Type Exposure Layer / API Gateway
Implementation Layer / Integration
Architecture
1. User Experience centric Mashup Yes No
2. Intermediation/Customization
Yes Not advised
3. Aggregation/Arbitration
Yes Not advised
4. Process Integration No Yes
SOA Services
SOA Middle-ware (ESB)
Back end
Back end
API
API Services
Back end
Back end
Adapter
DeveloperManagement
APIIntelligence
SOAManage-
ment
API API
Back end
API
DIGITALBUSINESSPLATFORM
© 2014 Apigee Confidential – All Rights Reserved
• The need for higher speed and agility demands for a new layer / system in the Enterprise IT
• The Digital Business Platform optimizes access for a large amount of different devices and an increasing number of business partners from the outside of the Enterprise
• It has to be deployed in a fault tolerant, high available, horizontally scalable manner to be prepared for the load of the near future
• It complements existing capabilities in the architecture:
– Services exposed by Middleware / SOA
– (Big) Data platform for data analysis / data mining / machine learning
Summary
23
17 July 2014
APIs are Interaction, not just Integration
S
© 2013 Apigee Confidential – All Rights Reserved
Agenda
25
InteractionExplosion of devices - User centric
DevelopersEmpowered developer - Usability is key
IntegrationUbiquitous - Cheap and easy - Integration Explosion
Interaction
© 2013 Apigee Confidential – All Rights Reserved
Partner Integration
Web site
Wearable Electronics
Gaming Consoles
Smart Appliances
Smart Sensors
Smart phones
Connected Cars
Tablets
KiosksAPI
API
Your Digital Assets
AP
I AP
I
27
Your Digital Assets
App and Partner Interaction take Centre Stage
© 2013 Apigee Confidential – All Rights Reserved
Interaction Patterns Govern APIs
Backend Services
App Server
Web
Partner ESB
Mobile API A
API B
API C
API Tier
© 2013 Apigee Confidential – All Rights Reserved
Interaction Patterns Govern APIs
Backend Services
App Server
Web
Partner ESB
Mobile API A
API B
API C
API Tier
Chattiness
Mash Ups and Aggregation
Data & FormatFeedback Loop
(email, push notification)
Security
Developers
© 2013 Apigee Confidential – All Rights Reserved
Developers are the Key Actor
31
DeveloperUser API API Team BackendApp
© 2013 Apigee Confidential – All Rights Reserved
This applies …
32
Jakob's Law of the Internet User Experience:
Users spend most of their time on other sites. This means that users prefer your site to work the same way as all the other sites they already know.
Developers spend most of their time developing against other APIs. This means that developers prefer your API to work the same way as all the other APIs they already know.
33
34
Application developers are raison d'être for APIs.
35
Be pragmatic.
For the benefit of application developers.
Design – intuitive and simpleConsumption – ease of use and relevancePackaging – developer and use case centricSecurity – simple and consistent
© 2013 Apigee Confidential – All Rights Reserved
API Design & Pragmatic REST
37
• https://groups.google.com/forum/#!forum/api-craft• http://info.apigee.com/Portals/62317/docs/web%20api.pdf• http://www.youtube.com/user/apigee
APIs are Integration Technology
© 2013 Apigee Confidential – All Rights Reserved
APIs – Easy Integration
39
Complex and VisibleCORBA, RMI, RPC
Simple and AbstractEAI, EJB, ESB
Simple and VisibleREST/HTTP
Integration is hard however System Impedance and
Remote Nature is obvious
© 2013 Apigee Confidential – All Rights Reserved
APIs – Easy Integration
40
Complex and VisibleCORBA, RMI, RPC
Simple and AbstractEAI, EJB, ESB
Simple and VisibleREST/HTTP
Integration is easier System Impedance
and Remote Nature is hidden
© 2013 Apigee Confidential – All Rights Reserved
APIs – Easy Integration
41
Complex and VisibleCORBA, RMI, RPC
Simple and AbstractEAI, EJB, ESB
Simple and VisibleREST/HTTP
Integration is easier System Impedance
and Remote Nature is hidden
Abstractions break!
© 2013 Apigee Confidential – All Rights Reserved
APIs – Easy Integration
42
Complex and VisibleCORBA, RMI, RPC
Simple and AbstractEAI, EJB, ESB
Simple and VisibleREST/HTTP
Easy integration based on simple,
standardised principlesUsed
everywhere!
© 2013 Apigee Confidential – All Rights Reserved
App
Backend
APIs are Everywhere
API
© 2013 Apigee Confidential – All Rights Reserved
App
Front end
Backend
APIs are Everywhere
App
Backend
API
API
© 2013 Apigee Confidential – All Rights Reserved
App
Front end
Backend
AppApp
APIs are Everywhere
API
API
APIAPI
© 2013 Apigee Confidential – All Rights Reserved
Front end
Backend
AppApp
Service Service Service
Service Service Service
Service Service Service
Micro service architecture
AP
Is
AP
Is
APIAPI
API
API
APIs are Everywhere
APIs not an Integration Platform
© 2013 Apigee Confidential – All Rights Reserved
• Every Developer is an Integration Developer– Bye bye centralized integration platform
– Hello point to point integration
• Integrations are use case centric– Bye bye canonical data model
– Bye bye one size fits all interaction model
What’s the consequence of ubiquitous APIs?
© 2013 Apigee Confidential – All Rights Reserved
Front end
Backend
AppApp
Service Service Service
Service Service Service
Service Service Service
Micro service architecture
AP
Is
AP
Is
APIAPI
API
API
The API Landscape will shift!
Requires• Making APIs consumable
and accessible• Adding governance
© 2013 Apigee Confidential – All Rights Reserved
Add Visibility, Governance and Decoupling
Backend Services
App Server
Web apps
Social apps
ESB
Mobile apps
Persistence
Security
Orchestration
Analytics
All Apps API Tier
© 2013 Apigee Confidential – All Rights Reserved
Wrap Up
51
InteractionExplosion of devices - User centric
API DesignEmpowered developer - Usability is key
IntegrationUbiquitous - Cheap and easy - Integration Explosion
Your Questions?
Thank you
17 July 2014
Data: Powering the Digital Economy
Pankaj Sodhi, Accenture UK/I Big Data Lead
@AccentureDigiUK
Every business is a Digital Business
Data is the lifeblood of the organisation
Most business leaders – CEO, CMO and CFOs now view data as among their most valuable assets
Copyright © 2014 Accenture All rights reserved. 57
Data Supply Chain
Data should flow easily and usefully through the entire organisation
80% of high performing companies are investing in analytics, yet only 40% day they are getting the value that they want
Application
Service Interface Layer
In-MemoryIMDB, CEP,
CacheIMDB, CEP Distributed Cache In-Memory DB
Streaming CEPGraph Stream
1 Pass Stream
Big Data Core
Query Engine
In-Memory Analytics
InteractiveBulk Transfer
Sources
Streaming
Organisations can choose from different data technology components to build the architecture needed to support data acceleration. These include big data platforms, complex event processing, ingestion, in-memory databases, cache clusters, and appliances.
Decision
Learning
Discovery
Memory
Perception
Optimization
Prediction
Causality
Tagging
Filtering
Raw Data
Actionable Insights
Data Ingestion Capabilities
Data Discovery & Visual Analytics Capabilities
Advanced Data Analytics Capabilities
Known Business Questions
Business Hypothesis
Data Raising Interesting Questions
Insight Generation
Patterns in the Data
Operational Analytics Opportunities
Accenture Data Discovery
Accenture’s Data Discovery Platform
Advanced VisualizationsGet deeper insights from analytics by using interactive best practice advanced visualizations that
enable drill-to-detail. For maximum effectiveness use these visualizations in combination with blended data and advanced analytic models.
Text & Unstructured
Segmentation/Clustering
Predictions & Simulations
Behavior
Networks and Relationships
Geospatial
Treemaps and scatter plots visualize aggregate and individuals size, proximity
and concentration of clusters
Parallel coordinates visualize patterns in multivariate discrete and continuous datasets with parameterized colors
Visualize location and geographic network data on multi-layered maps give insights
into event concentrations.
Parameterize time-series predictions for ‘What-if’ analysis. Confidence bands
visualize the models strength
Find relationship clusters and patterns with network diagrams are used chord charts for
hierarchical entity relationships
Text clouds combined with sentiment scores are powerful for generating insights from
social and unstructured data.
Thames Water
Copyright © 2013 Accenture All rights reserved.
@AccentureDigiUK
Join the Conversation
April 10, 2023
Apigee Insights
Who we are
Helps business quickly achieve digital acceleration, with scale, insights and agility
More API deployments run on Apigee than any other platform
Customers include 20% of the Fortune 100, 50% of the top global brands, 50% of the top retailers
Customer-driven product leadership with APIs and Big Data
Over 100k developers in the fast growing Apigee developer community
Products
Services
Knowledge
Apigee customers span multiple industries
66Updated 5/31/14
Agenda
Predictive Analytics on Big Data, enabling a new
generation of businesses
Customer Success
Apigee Insights
Enterprises need to operate differently to stay relevant
68
Omni-channel AgilityIndividualization
Individualized interactions
Relevant experiences
Accelerate decisions
Proactive response
Adaptive processes
Continual learning
Be where customers are
Holistic view
Contextual journey
Proliferation of Mobile and Digital
Accelerating Pace of Change
Evolving Customer Expectations
Predictive Analytics on Big Data key to operating differently
69
DeveloperUser API API Team Backend
Predictive InsightHadoop
Data Warehouse
AppAdaptive App
Data Scientist
Adaptive API
Existing Big Data investments lack key capabilities
70
Data Lake / Data Warehouse
• Big Data Infrastructure
• Lack specialized data structures to analyze fine-grain event data
• Lower precision resulting from predictive analytics on summarized, tabular data
• Difficult to incorporate signals from unstructured data in analysis
• Difficult to access insights in real-time
• Difficult to integrate predictive insights into apps and APIs
Business Intelligence
• BI Visualization & Reporting
Data Science • Statistical Analysis• Scientific Visualization
ChallengesCurrent Big Data Investments
71
In addition, many challenges to address
Best Practices
Global Deployment
Unstructured Data Processors
Real Time Processing
Continual Improvement
APIs & Automation
Sophisticated Machine Learning
Model Building
Monetization
Security
System Management
Big Data Structure
AdaptiveApps & APIs
Agenda
Predictive Analytics on Big Data, enabling a new generation of businesses
Customer Success
Apigee Insights
Business & Technical
UserRapidly realize value API Team
Apps
Developer
Develop more precise insights
Discover complex hidden patterns
Data Scientist
Apigee Insights: Enable adaptive apps and APIs
Hadoop Data Warehouse
Other
Developer & Business User Tools
Predictive & Descriptive Analytics
Graph & Sequence Processing (GRASP)
Business User Tools
Developer Tools
RESTful APIs
Graph Machine Learning
Modeling Workbench
Real Time Scoring
Data LoadersUnstructured Data
ProcessorTemporal Graph
Database
Apigee Insights: Enable adaptive apps and APIs
Business & Technical
UserAPI Team
Apps
Developer
Data Scientist
Hadoop Data Warehouse
Other
Predictive & Descriptive Analytics
Developer & Business User Tools
Graph & Sequence Processing (GRASP)
How do you detect hidden patterns and relationships?
75
Data Data
• How do you discover hidden patterns in
interactions across touch points & channels?
• How do you understand the customer journey
across siloed channels?
• How do you deliver a contextual journey vs.
channel optimized interactions?
• Unique big data structure (GRASP) purpose-built to analyze fine-grain time-aware interaction data – at scale
• Identify complex hidden patterns and relationships in customer behavior
• Dramatically improve ability to predict future actions
Use specialized big data structure to analyze customer journey
76
PROFILEConsumerID: U56Gender: MGeo: San FranciscoInterests: Bikes
PROFILEConsumerID: U60Gender: FInterests: NewsAge: 35-40
2 4
1 3 40
0 3
View product A
on web
Buy product A on mobile
Call customer service
Return Product A in store
3
2
4
1
0
Time
GRASP (Graph and Sequence Processing)
time-sequenced graph database on Hadoop
Fine-grain event and entity data
How do you improve prediction accuracy?
77
• How do you avoid losing precision when working
with sampled or summarized data?
• How do you detect signals hidden in
unstructured data such as text?
• How do you prevent prediction accuracy from
degrading over time?
Apply powerful machine learning on fine-grained event data
• Significantly improve prediction accuracy by applying powerful machine learning algorithms directly on fine-grained event data in GRASP
• Always deliver fresh, relevant interactions with a continual learning feedback loop
78
Rules Based
Approach
Traditional StatisticalAnalytics
Apigee Insights
Precision
Traditional Analytics Apigee Insights(Machine Learning on Big Data)
Static Model Dynamic Model
Focuses only on top attributes
Leverages both strong and weak signals
Batch only Batch and real time
Adapts to changing conditions
Easier to put into action
Significantly better results
Discover hidden signals in unstructured data
79
MBRS payment was rcvd the same day as
dunning took effect. Dunning was not posted until
after payment was processed. Final notice sent
out after payment was sent out as well. Payment was sent
out and postmarked before due date. The member
was termed correctly.
88%
45%
With Unstructured
Data
Without Unstructured
Data
Prediction Accuracy(Likelihood for member to complain)
Healthcare Payer Example
2x
• Unstructured data is difficult to analyze but holds valuable signals
• Improve prediction accuracy by extracting signals hidden in text
How do you realize value from data?
80
• How do you enable developers to build new
adaptive apps and APIs?
• How do you enable business users and analysts
to interact with predictive insights?
• How do you integrate predictive insights into
existing apps and systems?
Predictive
Insights
Business Value
Access insights in real-time
81
• In-memory real-time processor to enable fast access to insights
• Regularly updated to always provide fresh, relevant scores
Real-time access to insights using NoSQL
Propensity Free Shipping 10% Off Churn
User 1 0.72 0.68 0.33
User 2 0.56 0.23 0.55
User 3 0.32 0.45 0.67
User 4 0.20 0.32 0.18
User 5 0.44 0.69 0.22
Easily consume insights via APIs and visualization
82
Predictive insights
Descriptive insights
• Use RESTful APIs to build new adaptive apps and APIs or extend existing ones
• Enable developers and business users to act on predictive insights
Build new adaptive apps and APIs, extend existing ones, or use or extend visualization tools provided
Deliver individualized interactions across multiple channels
83
Direct Mail
Web
Mobile
Outreach
Fine grained event data
Individualized interactions across multiple channels
Machine Learning on GRASP
Predictive Models
Targeting Recommendations
Churn Other Advanced
3
2
4
1
0
Propensity Free Shipping 10% Off Churn
User 1 0.72 0.68 0.33
User 2 0.56 0.23 0.55
User 3 0.32 0.45 0.67
User 4 0.20 0.32 0.18
User 5 0.44 0.69 0.22
0 View product page on web
1 Receive mobile offer
2 Call customer service
3 Agent note: dissatisfied
4 Return product in-store
5 Purchase Online
Customer Profile
Agenda
Predictive Analytics on Big Data, enabling a new generation of businesses
Customer Success
Apigee Insights
85
IBC: Proactive Customer Service with Predictive Analytics
“Apigee has helped us move from simply answering customers’ calls to proactively reaching out to our members before they have issues.”
Somesh Nigam, SVP and Chief Informatics Officer, Independence Blue Cross
• $1.2B Medicare Advantage business
• 50+% Reduction in complaints to Medicare
• Transformed from reactive to proactive customer service
• Identified root causes for dissatisfaction
Typical Use Cases
Proactive Churn Reduction
Reduce churn by predicting customers likely to be at-risk and proactively responding to them
Individualized Targeting & Recommendations
Improve customer retention, cross-sell, up-sell, loyalty, and acquisition
20% increase in profit
Individualized daily deals emails
Leading Online Services Company
100% increase in revenue & profit
Individualized website recommendations
Leading Shopping Site
164% lift in conversions
60% more clicks, 55% fewer emails
Recommended products - game launch
Leading Gaming Company
50% fewer complaints
6x more precise predictions
1 month in advance
Leading Healthcare Insurer
Apigee Insights
87
Deep Predictive Analytics Expertise
• Team led Big Data @ Yahoo!, IBM, and others
• Thought leaders in machine learning and big data
• Extensive Vertical expertise from client engagements
Accelerate Business Growth
• Increase revenue, profitability
and customer satisfaction
• Individualized targeting and
recommendations
• Proactive Churn Reduction
Develop Adaptive Apps & APIs
• GRASP on Hadoop
• Machine Learning on GRASP
• Unstructured & API data
• Real Time
• APIs and Automation
• Cloud or on-premiseApigee Insights
Thank you