Analytics on system z final

© 2015 IBM Corporation

zAnalytics DB2 Update Day, March 2015

1

DB2 Update 2015



2

Status 2015

� Iceland, Reykjavik, Nyherii 13 17-3-15

� Finland Helsinki, IBM 51 23-3-15

� Norway, Oslo, Evry 59 24-3-15

� Sweden, Stockholm, SEB 118 25-3-15

� Denmark, Copenhagen, DB 109 26-3-15

� Denmark Silkeborg, SD/JB 53 27-3-15

� Total 403

DB2 for developers, architects, DBAs and other good peoplewho really appreciate the strongest relational database in the world.



3

Agenda

DB2 Use Cases - Trends and directions

coffee

DB2 Archives / DB2 Table Archives and others – comparison

Managing DB2 with IBM Admin & Compare for System z / Update on functionality & Customer UseCase

Lunch

DB2 11 Update for Developers

coffee

Native Stored Procedures / SQL/PL for DB2 11 / Revisiting

Native Stored Procedures & SQL/PL

Closure



4

IBM z Systems Mobile Enablement

Focus on Information and Analytics

Peter SchouboeIBM zAnalytics, [email protected]



5

Agenda

� Mobile affects IT and need for Analytics

� Some infrastructure notes on z Systems

� Analytics on z Systems

– Reporting

• QMF

• Cognos

– Forecasting

• SPSS

– Decision support

• ILOG

� Summary



6

Mobile pervasiveness is demanding higher service levels

6

91% 75% 96% 90% 900%mobile users keep their device within arm’s reach 100%

of the time

mobile shoppers take action after

receiving a location based message

year to year increase in mobile

cyber Monday sales between 2012

and 2011

users use multiple screens as

channels come together to

create integrated experiences

increase of global machine-to-machine connections by 2022

(2 billion in 2011 to

18 billion at the

end of 2022)

© 2013 IBM Corporation6



7

Mobile changes the business landscape

Finance & BankingManage their investment portfolios and accounts from anywhere for complete bank transactions

InsuranceFile, process and manage claims and document damages as they happen

Travel & TransportationProvide up to date information specific to their itineraries and location and enable customer self-service

Construction & ManufacturingManage complex projects

and operations on site and streamline survey

and work order processes

RetailEngage shoppers in new

ways and intelligently target personalized and location sensitive marketing offers

Cross-Industry CIO’’’’s OfficeEmpower employees with anytime,

anyplace access to dashboards and critical information

Mobile data traffic will grow at 61%(CAGR) 2013 to 2018Where will it be stored ? How will it be manage & processed?



8

Mobile computing driving real-time and 24/7 business analyticsz Systems Hybrid Transaction and Analytics Platform supporting the mobile experience

� Business execs need to know the health of their business 24/7/365

� The “mobile office” supporting timely,

confident decision making - delivering

business insight any time any where

� Mobile drives real time business analytics

� Customer experience influenced by content -geo-specific offers and service offers ‘as-you-need them’ e.g. while consumers are in the store (the “mobile-moment”)

Systems of engagementneed a secure,

scalable, always-on

System of Record



99



10

Mobile workloads impact systems of record differently than web workloads.

� Increase in off-peak transactions. Expect 10-50% growth as you add a mobile channel.

� Increased query or “read-only” transactions. As many as 50% of mobile transaction could be “read-only”.

� Unanticipated spikes in workload due to popular apps, features or special offers. No traditional times for workload spikes.

� Inefficient applications written by “non-professional” coders. Drives up transaction rates unnecessarily.



11

Agenda




– Reporting

• QMF

• Cognos

– Forecasting

• SPSS


• ILOG

� Summary



12

Complex IT infrastructure Issues- performance issues, delays, errors, hard to debug, expensive to maintain…..



13

System zPR/SM

Hipersockets

Crypto functions

zSystems and Analytical Application Workloads on zLinux.

CP

U

CP

U

CP

U

zVM

z L

inu

x

z L

inu

x

z L

inu

x

z L

inu

x

z L

inu

x

z L

inu

x

z L

inu

x

z L

inu

x

DB2 UDB Cognos

IFL

IFL

IFL

IFL

IFL

IFL

SPSS Data Mart

zIIP

CP

U

CP

U

zIIP

zIIP

LP

AR

1z/O

S

LP

AR

2z/O

SAnalytical Workload

Operational BI

DB2/z IDAA link

CICS CICS

Transaction Workload

• Unlimited number of Linux clients

• 100% utilization on IFL

• Shared everything(Memory, Discs, CPU)

• No Network latency

• IFL’s on max capacity

• Box upgrade includes new IFLs – at no cost

• Only zOS CPs counts in MLC/OTC

• zIIP offload on DB2 Usefrom zLinux

IDAAHTAP Query



14

* No server can fully exploit its maximum I/O bandwidth

PCI – Processor Capacity Index (IBM MIPS)

141-wayCustomer Processors

PCI for1-way1695

Memory10 TB

System I/O Bandwidth832 GB/Sec*

80-way

64-way

54-way

1.5 TB

512 GB 1202902600

288 GB/sec*

172.8 GB/sec*

1514

3TB

384 GB/Sec*

101-way

z10 EC

z9 EC

z196

zEC12

z13

IBM z13: Advanced system design optimized for digital business



15

• Right-time insight at point of impact

• Increased business agility

• Improved data governance and security

• Reduced infrastructure cost and complexity

Transactions

Analytics

And - a new Strategy for the new type of Analytics is required….IBM/Gartner named it Hybrid Transaction-Analytical Processing (HTAP)

GARTNER:

“By eliminating analytic latency and data synchronization issues, hybrid transaction/analytical processing will enable IT leaders to simplify their information management infrastructure”

“This architecture will drive the most innovation in real-time analytics over the next 10 years via greater situation awareness and improved business agility”

Gartner Research Note G00259033: Gartner 01-2014 Hybrid Transaction Analytical Processing Will Foster Opportunities



16

DATA

Transaction Data

Customer Data

Account Data

Payment Data

Claims Data

System z Host:

System zOrchestration

Analytics: Predictive & Descriptive

Business Rules

networknetwork

Key Characteristics

DATA

Transaction Data

Customer Data

Account Data

Payment Data

Claims Data

System z Host:

System z

Orchestration

Analytics: Descriptive & Predictive

Business Rules

WHY? • System z already hosts a majority of the systems of

record data that feeds business analytics

• Unparalleled, proven performance execution of combining transactional data environments, analytics & rules

• Best of breed security for sensitive data

• Extensive logging for data and transaction governance

• Existing comprehensive, tested HA / DR capabilities

• Cost savings from reduced systems, software & people

• Plug and play capacity extensions and migrations enabling quick production application deployment with no downtime

High Level Strategy: Bring Analytics to the Data

� Much operational data resides on z/OS

– Many customers ETL this data to other platforms for analytics

– ETL: Expensive, error prone, security concerns, data not current, multiple copies

� ETL can be avoided

� Integrated Transaction and Analytics Processing

– OLTP and analytics in the same database



17

IBM DB2 Analytics AcceleratorEnhanced with PureData System for Analytics N3001

Benefits

� Extreme performance for complex queries

- Up to 2000x performance improvements

� Cost Savings

- Offload complex queries and eliminate costly query tuning

- No need to create or maintain indices

- Improves access to and lowers the cost of storing, managing and processing historical data

� Integrated with DB2 z/OS and inherits mission critical features such as security and recoverability

� Access to DB2 Analytics Accelerator is transparent to applications and users

� Fast deployment and time to value

- Installation is non-disruptive

- Plug it in, load data and go in 1-2 days

A high performance appliance that integrates Netezza technology with

zEnterprise technology to deliver dramatically faster

business analysis

Highlights – What’s New?

The PureData System for Analytics N3001 provides additional benefits to DB2 Analytics Accelerator customers:

� Advanced security through encryption of data at rest with self-encrypting disks

� Performance improvements for analytic workloads

� Improved serviceability with the recently introduced automatic call home capability

17



18

DB2 Analytics Accelerator Roadmap

ISAO V1

Query Accelerator

Storage Saver

Advanced Analytics

Unified Store

2011

2012/2013/2014

SPSS & HADOOPIntegration integration

Accelerating in-database

transformation

Sophisticated multi-temperature data and archiving solution

• Enhancing current capabilities

• Enabling more query acceleration

• Increasing transparency

• Supporting new use cases

• Utilizing PDA technology advantages

IDAA V2.1 (11/2011

•Netezza acceleration

IDAA V3.1 (11/2012)

•Incremental Update

•Storage Saver

IDAA V4.1 (11/2013)

•Static SQL

•Workload balance

IDAA V4.1 PTF 5 (04/2015)

•In-database transformation

•Incremental Update Turbo

IDAA V4.1 PTF 4 (10/2014)

•Encryption of data at rest

•Call Home

ELT Accelerator

2015



19

How we enable Mobile & Analytics with DB2

–Inside engine

• SQL Performance improvement

– (v11 provides 10 – 40%)

• SQL Enhancements (SIMD..)

– & SQL Optimization tools

• Native Stored procedures (zIIP)

• Big Insight support / JSON

– Hybrid platform enrichement

• DB2 Analytic Accelerator

• Java & .Net interfaces (zAAP)

• DRDA support (zIIP)

– Enabeling Business Applications

• Porting Analytics Apps to System z and zLinux

• Rational support



20

� Database Optimization

– DB2/z hardware utilization

• Compression / Encryption / Floating Point calculations

• zIIP eligible workload / e.g. distributed load through enclaved services

– z Systems Memory and performance enhancements

• DB2 Bufferpool enhancements (more in every version)

• The more memory – the lower CPU usage for queries

• Sysplex & data sharing – data ALWAYS available and unlimited growth available

• DB2 Version 11: 10 � 40% lower CPU for DB2 workloads

� Database New Functionality

– DB2/z near real time data for analytics – run off z platform – inside DB2.

• DB2 Analytics Accelerator

– DB2/z Mobile App enablement

• SQL enhancements (Cube, Rollup, Range etc..) (SMID supported)

• Mobile (JSON), Java (JDBC), Hadoop (BigInsight)

• Native Stored Procedures allows for optimized Business Logic in Mobile Apps

� Application Support

– zLinux

• Application Hub inside z – outside zOS

• Sharing zSystems strengths for RAS / allowing openness and flexibility of Linux

– z/OS

• Porting new Analytics Applications to optimize workload for mobile

Primary z Systems & DB2 updates enabling HTAP



21

Impressive DB2 11 Performance Results!

Query

Batch

OLTP

XML



22

Agenda




– Reporting

• QMF

• Cognos

– Forecasting

• SPSS


• ILOG

� Summary



23

Degree of Complexity

Co

mp

etitive

Ad

va

nta

ge

Standard Reporting

Ad hoc reporting

Query/drill down

Alerts

Forecasting

Simulation

Predictive modeling

Optimization

What exactly is the problem?

How can we achieve the best outcome?

What will happen next?

What will happen if … ?

What if these trends continue?

What actions are needed?

How many, how often, where?

What happened?

Based on: Competing on Analytics, Davenport and Harris, 2007

Reporting

Stochastic OptimizationHow can we achieve the best outcome including the effects of variability?

Descriptive

Prescriptive

Predictive

Business Analytics Adoption Landscape

Increasing prevalence of compute and data intensive parallel algorithms in commercial workloads driven by real time decision making requirements and industry wide limitations to increasing thread speed

Analytics

Operational

Decision Management

QMF

Algorithmics



24

QMF (Query Management Facility) / DataQuantIBM’s most ‘simple to use’ tool, for SQL driven reports

QMF



25

QMF



26

QMF Websphere for zLinux

QMF

QMFWebsphere



27

QMF™ for z/OS 11 goes where you go

Go Mobile!!

QMF for z/OS allows your users to access QMF dashboards and reports from their tablet devices…Build once, deploy everywhere.



28

Cognos Analytics on z� Strategy:

- Cognos – largest IBM’s acquisition

� Mainframe Business

- Continuous success of large client base on most mature platform, with half

century history

� Technology trend:

- Analyze the data where it resides

� Cognos 8.4 launch on Linux on System z (2008)

Consistency - since 2008, every Cognos release on Linux, Unix and Windows was

available on Linux on System z

� Cognos 8.4.1 on z/OS – Unix System Services (USS) (2011)

Delay due to technical complexity of porting to USS compared to Linux on System z

� Cognos 10.2 on USS (2012)

� Cognos 10.2.1 on USS with Mobile Release – USS (2013)



29

IBM Cognos Business Intelligence (for Linux on System z & z/OS)

Provides the breadth of BI capabilities required to supports all users from business critical to tactical to strategic

� Provides the qualities of service required for business critical analytics

� Enables organizations to deliver BI as a service

� Provides access to hadoop data in place

� Reduces the cost of standardization/consolidation



30

IBM Cognos and QMF BI for the Mobile Workforce

� Experience insight wherever you are

– Uninterrupted productivity with quick andeasy access to analytics anywhere you go

� Interact with information offline or online

– Rich, visual and interactive experience

� Confidently and easily deploy Cognos BI on industry leading form factors

– Single authoring and administrative

environment to deliver secure, relevant, and reusable content

On the road or in the office –same great insight.

Simple, reliable, and secure.



31

IBM Business Analytics for z Systems

� Understand your current state and your potential state

� Monitor results and fine-tune your business� Inform strategy with a view into the future

Cognos BI for zEnteprise

Business intelligence

Business outcomes/benefitsBusiness analytics capabilities

� Reporting, analysis, operational & financial planning and consolidation.

� Product profitability solutions across customers, business lines and channels

� Sales Performance Management to improve pay-for-performance programs and efficiency in incentive compensation processes

TM1 for zEnterprise

Performance management

SPSS for zEnterprise

Predictive analytics

� Predict customer segment and category affinity� Market Basket Analysis to identify the next

best offer for the customer� Overlay browsing history onto purchase history

to profile customers



32

Introduction to COGNOS - Core Architecture



33

FAST: In-Memory Analysis using Dynamic Cubes

Easier Sizing process

with the Cube Designer Hardware Sizing Guide

(for more details attend the session



34

What is predictive analytics?

Predictive Analytics helps connect datato effective action by drawing reliable conclusions about current conditions and future events

Gareth Herschel, Research Director, Gartner

Group

SPSS for zEnterprise

Predictive analytics

� Predict customer segment and category affinity� Market Basket Analysis to identify the next

best offer for the customer� Overlay browsing history onto purchase history

to profile customers



35

What does a predictive model do?

Cust:ID Balance Monthly savings > 1 Account? Account type Churn

XY6733 1.000 25 Y GT Y

XY7766 50 0 N GX Y

XY6079 5.000.000 500 Y GT N

HH4455 525 100 N GXD N

1. Collect historical information on customers that churned, and customers that stayed

2. Model searches for common characteristics

3. Model generates rule set that best predicts churn

Rule Set

Churn is likely, when:1. xxxxxxxxxxxxxxxxx

2. xxx xxxx

3. xxx xx xxx xxxx

Customers at Risk

1. xxxxxxxxxxxxxxxxx

2. xxx xxxx

3. xxx xx xxx xxxxx

Y

Y

Y

Y Y

YY

Y

Y N

N

NN

N

N

N

N

N

N

NN

N

N

N

N

N

N

N

N

N

NNN

NN

N

N

N

N

Var A

Va

r B



36

Technique Usage Algorithms

Classification(or prediction)

• Used to predict group membership (e.g., will this customer leave?) or a number (e.g., how many widgets will I sell?)

• Auto Classifiers, Decision Trees, Logistic, SVM, Time Series, etc.

Segmentation • Used to segment data points into groups that are internally similar and externally dissimilar.

• Identify cases that are unusual

• Auto Clustering, K-means, etc.

• Anomoly detection

Association • Used to find events that occur together or in a sequence (e.g., market basket)

• APRIORI, Carma, Sequence

SPSS ‘Technical’ - Data mining techniques

Association ClassificationSegmentation



37

Predict at the point of impact!

Level Points

Low Risk > -5

Medium Risk > +1

High Risk > +8

Predictive Analytics

Rules

Structured, Unstructured, Social Media. & Business Intelligence Data

Simulation & Optimization

Scoring

Front-line Rep only sees “Refer” at the point of impact



38

Taking Real-Time Scoring to the next level with z System

DB2 for

z/OS

Application w/latest dataReal-Time Score/

Decision Out

ETLData In

R-T, min, hr, wk, mth

Reduced Networking

In-transaction & In-database scoring on the same platform

Consolidates Resources

Meet & Exceed SLA

DB2 for z/OS Data

Historical Store

Copy

SPSS Modeler

For Linux on System z

Scoring Algorithm

Business System /

OLTP

� Improves the accuracy of Predictive Analytics by…� Scoring new and relevant

data directly within transactional applications

� Scaling to large data volumes for use in building data models

� Score 12K transactions per second in real-time

✔✔

✔

✔

✔



39

SPSS Predictive Analytics (for Linux on System z)

SPSS Statistics SPSS Modeler Gold

�Applies math to decision making and

research for commercial, government,

and academic users

�Data mining tool that generates hypotheses

and scoring

�Provides text analysis for unstructured data

to model consumer behavior

� Includes In-Transaction Scoring with DB2

z/OS: embeds the scoring algorithm directly

within the transactional application



40

Business Decisions are Everywhere

We need to add an eligibility check to

meet the requirements of the new regulation.

Let’s create a special promotion for our best

customers.

Can we automate approvals for this type

of order?

And They Change Frequently

Benefit calculation

Fraud assessment

Eligibility Pricing

Underwriting

Configuration

Tax calculation

Billing

Product Selection

Accounting Disposition

Compliance Screening

Documentation Requirements

Commissions / RoyaltiesUp-sell/Cross-sell offer



41

Externalize Decisions from Applications into Rules Manage decision logic independently from applications

Business Rules

ApplicationApplication

Decision logic

� Natural language rules can be easily read

� Externalized rules are easy to change

� Centralized rules enable reuse and consistency

� Rules written in software code cannot be read by business people

� Hard coded rules are difficult to change

� Rules intertwined within applications cannot be reused by other systems

Without Decision Management With Decision Management

41



42

Redefined Application Change Cycle

DeveloperBusiness - IT

Weeks / MonthsDays / Weeks

Functions / Tasks / FlowDecisions / Policies



43

IBM Operational Decision Manager v8.0.1

Rule DesignerEvent Designer

Rule Solutions for Office

Decision Center Versioned Assets

Rule Execution Event Execution Decision Monitoring Connectors

Manage

Decision ServerConsole

Design Monitor

Decision Server

Deploy Measure

VisibilityCollaborationGovernance

Define Update

Web Services – API - GUI

DevicesEnterpriseApplication

POS BPM CRM

Social

Event Widgets Space Business Console

Enterprise Console

Access and ControlDecision Artifacts

IBM Operational Decision Manager



44

Manage Decisions at the Speed of BusinessMajor system updates are not required for decision logic changes

requests response

Business Rule Updates

� New pricing promotion (monthly)

� Updated claims policies (quarterly)

� Tightened regulations on trading (annually)

Approve filed claim

Detect fraudulent trades

� Customer searches for a price

2

Decision Logic

� Insured members submit claims

� Brokers make trades

1

Determinepricing & bundling

� Grow customer base

� Increase customer satisfaction

� Comply with regulatory changes

Why Decisions Change?

Application

Business Rules



45

What does ODM bring to z/OS?

Challenges forMost z Clients

1. Consolidation, Isolation, Extension or Extinction of COBOL application portfolio

2. Be able to react to increasing variety and volume of change requests

3. Sharing business rules across platforms & channels

4. Ensuring seamless business experience in migration/ application evolution

Benefits of the ODM Approach

� Cost savings

– Shorter change cycle, without increased business risk

– Rule engine processing is zAAP/zIIP eligible

� Improved agility

– Improved Time to Market

– Manage business decisions in natural language

– Decouple development and business decision change lifecycles

� Single version of the Truth

– Consolidated and shared expression of business policy

– Maintainable with a Center of Competency model

� Incremental Adoption

– Deploy decision methodology one decision at a time

– Focus on decisions that need to change often & quickly

– Expand adoption of “market validated” decisions



46

IBM Decision Server

Leverage a wide range of platforms to meet the varying needs of enterprise architectures

IBM Decision Center(Windows, AIX, Linux, Linux on System z, z/OS)

Multiple z/OS deployment options

Windows AIX Linux (x, p & z) z/OS

Platform support

Repository

IBM Operational Decision Manager



47

Agenda




– Reporting

• QMF

• Cognos

– Forecasting

• SPSS


• ILOG

� Summary



48

z Systems: vision, strategy and technology to fuse transactions and analytics

Streamlined decisioning process

Faster transactions

Faster transformationof data

Faster, more accurate reporting

Faster, more accurate modeling

Cognos

QMF

(IDAA)

InfoSphere

Repl. Server

Information Server

CICS, IMSWebSphere,

SAP, …

Easier integration of other data sources

ODMBPM

(ILOG)

InfoSphere

BigInsights

InfoSphere

Connector

Faster, more accurate scores

Best of class integrated data life cycle management for: Generating customer

insights, fighting fraud, preventing financial crimes, …

SPSSCognos



49

The z Systems differential

Mainframe continues to be more adaptive and affordable for mobile workloads

IBM announces new pricing : UP TO 60% reduction of processor increase due

to mobile workloads (no LPAR isolation) and also OTC VUE options

FASTESTINSTANT Response

Fastest response time, real-time access to

System of Record, current data - not old

copy. Acceleration of queries (up to 2000x)

to meet mobile

demands

TOUGHEST

Data security and resilence

Modern agile tooling with pervasive best-of-breed security, scalability, RAS

AVAILABLE

data 24/7

Only System z can handle the

massive spikes and growth of

mobile workloads



50

And don’t forget World of DB2

50

And IDUG, and Insight, and……

Analytics on system z final

Data & Analytics

mobile office

mobile users

ibm admin

ibm corporation6

mobile changes

mobile pervasiveness

mobile computing

system z update