Delivering Insights with Next-Generation Analytics–Generate ad hoc queries vs. any data source ... Sybase IQ Analytic DW (running on IBM p series ... Insights with Next-Generation

Beyond Reporting: Delivering Insights with

Next-Generation Analytics

Wayne EckersonDirector, TDWI Research

July 8, 2009

Sponsors:

Speaker

Wayne Eckerson

Director, TDWI Research

Agenda

• Definitions

• Types of users

– Analytical requirements

• New analytical tools

Definition

“Analysis is the tracing of things to their

source and the resolving of knowledge into

its original principles.”

-- Webster’s Dictionary

Business Definition

Iterative Q&A

Person to Person Person to Data

UpshotSegmentsPatterns

Associations

Root Cause

ForecastsAction

Trends

Analytical Challenges

• People

– Knowledge of process,

data, and tools

• Data

– Integrated, accurate,

trustworthy data

• Defacto tools

– Spreadsheets and

desktop databases

User Classification

80% of the Time 20% of the Time

Casual

Users

Task Tool Task Tool

MonitorMAD

Dashboard

Issue queries BI search

Analyze Discover trends Visual discovery

Drill thru Create plans Excel/Planning

Power

Users

Author Ad hoc query Monitor

MAD

Dashboard

Plan Excel/Planning Analyze

Analyze OLAP/Visual discovery Drill thru

Predict Data Mining

Tailored Ad hoc

Casual User Analytics

Power Users

Business Analysts

Analyst tasks

1. Gather data

2. Analyze data

3. Present data

Ranked toolsets

1.Spreadsheets

2.BI reporting tools

3.SQL

4.OLAP

5.Desktop databases

Super Users

Super user tasks

– Departmental ad

hoc reporting

– Extend MAD

– Participate in BICC

Ranked toolsets

1. BI reporting tools

2. Spreadsheets

3. SQL

4. OLAP tools

5. BI authoring tools

Analytical Modelers

Tasks

1. Gather data

2. Prepare data

3. Model data

4. Score data

Top five tools

1. Data mining

workbenches

2. BI reporting tools

3. SQL

4. OLAP tools

5. Advanced

visualization tools

New Analytical Tools

• BI Search

• Visual discovery tools

• Workgroup BI

• In-database analytics

• Analytical sandboxes

• Analytic engines

15

Ad hoc for Non-Technical Users?

•Simple

•Fast

•Comprehensive

16

BI Search

• Level 1 – Index documents in specific file formats

– Display reports by relevance; by category

– Suggest related reports

• Level 2 – Index metadata and generate SQL

– Generate queries against BI metadata and sources

– Generates charts; export to Excel, etc.

• Level 3 – Use natural language to generate SQL

– Generate ad hoc queries vs. any data source

– Generates charts; export to Excel

Example

Visual Discovery Tools

• Benefits

– Speed of thought analysis

– Interactive charting and visualization

– Flexible filtering

– Fast to deploy

– Less expensive

– Increasingly scalable

• Vendors

– Tableau, Tibco Spotfire

– SAS, Advizor Solutions

Territory

Central

Eastern

Western

($2,000K) ($1,800K) ($1,600K) ($1,400K) ($1,200K) ($1,000K) ($800K) ($600K) ($400K) ($200K) $0K

Quota Remaining

Nelson Zenarosa

Christophe Attiga

Sue Anthony

Vincent Vaghee

Select Salesperson to See Sales and Accounts:

Salesperson

Vincent Vaghee

Sue Anthony

Christophe Attiga

Nelson Zenarosa

01-Mar-08 01-May-08 01-Jul-08

Date Created

$0.0M

$1.0M

$2.0M

$3.0M

$4.0M

$5.0M

$6.0M

Actu

al V

alu

e

Sales to Date for Nelson Zenarosa:

$2M $4M $6M $8M

Estimated Value

Premium Cycle Works

National Bicycle Shoppe

Active works

Magnificent Cycles

Advance Cycles

Creative Cycle Shoppe

Bikes and only Bikes

Fear to ride Cycle Shop

Creation Supplies

Have a nice Deal Shop

Wingtip Cycle Works

Only Maintenance and Repa..

Cycles Stop

Roadside cycle shop

Era Cycle Shop

Xeta Cycle Works

Advance bicycle Shop

Cycling Shop

Alpine Cycle Shop

Loyal Bicycle Shoppe

Seventh Cycle Shop

Bikes and Accessories Shop..

Woodgrove Cycles

Leader cycle Shoppe

$1.49M

$2.02M

$2.16M

$2.19M

$3.00M

$3.20M

$4.49M

$7.61M

$1.45M

$1.52M

$1.85M

$5.83M

$1.45M

$1.56M

$1.82M

$1.85M

$2.17M

$3.25M

$3.39M

$6.89M

$1.60M

$1.81M

$1.84M

$2.14M

Accounts for Nelson Zenarosa:

Visual Analysis

Workgroup BI

• End-to-end SQL-based BI suites

• Single design environment

• Direct data access without semantic layer

• Web 2.0 interfaces

• Geared to workgroups and departments

• Affordable pricing

Vendors

• Startup vendors

– Inetsoft, LogiXML, eThority, Corda, iDashboards, Visual Mining, Autometrics, Gaussoft

• Visual discovery tools

– Advizor Solutions, Tableau, Tibco Spotfire, Qliktech

• BI vendors – SMB offerings

– SAP BusinessObjects Edge

– MicroStrategy Reporting Suite

In database analytics

External

Data

Data

WarehouseSource

Systems

Data Mining

Workstation

External

Data

Source

SystemsSource

Systems

Data Mining

Workstation

Data Warehouse

Data preparation

Data modeling

Data scoring

Data preparation

Data scoring

Data modeling

Traditional mining In-database analytics

In database analytics

Today In Three Years

Data Preparation 44% 60%

Model Building 17% 32%

Scoring 17% 32%

Percent of respondents who will use a database to perform

the following predictive analytic functions

Based on 166 responses, Wayne Eckerson, “Predictive Analytics: Extending the Value

of Your DW Investment,” TDWI Research, 2007.

Analytical Sandboxes

Data Warehouse(RDBMS)

Other

Data MartsAnalytic

Data Mart

Inside the DW

Data Warehouse(RDBMS)

Analytic

Data Mart

Other

Data Marts

Outside the DW

eBay’s Prototyping Environment for

Teradata (PET)

• “As quickly as we integrated renegade data marts into our data warehouse, new ones poped up. So we created virtual data marts in the EDW and user adoption has gone through the roof!”

• “We give users their own dedicated database to which they can add data and combine it with data from the EDW. They use these to prototype new analytical solutions. When they feel their solution is production ready they turn it over to development.”

• “Now the business gets the freedom to create the tables they want and they don’t have to worry about running a SQL Server on their desk.”

– Oliver Ratzeburger, Director of Information Management Architecture and Operations at eBay.

Next-generation sandboxes

Data Warehouse(RDBMS)

Other

Data Marts

Analytic

Data Mart

Cloud-based Computing

-No capital expenditure

-Quick to deploy

-On demand capacity

-Pay as you go

Analytic Engines

• Optimized for query processing

• Quick to deploy

• Good price/performance

• Linear scalability

• Fast performance

Categories

• Software-only products

– Columnar Databases• Vertica, 1010data, Exasol, InfoBright, ParAccel, Sand, Sybase

– MPP-only Databases • Aster, Datallegro, Greenplum, DB2, Kognitio, Teradata

• Hardware/software combinations

– Single vendor appliances• Netezza; Dataupia; IBM Infosphere Balanced Warehouse;

Teradata 550 and 2550;

– Multi-vendor appliances or “bundles”• Oracle-HP Database Machine; Oracle Optimized Warehouses;

Microsoft/Unisys, Sun/Greenplum DW Appliance, Sybase IQ Analytic DW (running on IBM p series)

Recommendations

TOOLS USERS

Visual Discovery Tools Individual business analysts

Super users building departmental

applications

Workgroup BI Tools Individual business analysts

Super users at SMB companies

In-database Analytics Analytical modelers + IT administrators

Analytical Sandboxes Business analysts + IT administrators

Analytic Databases Business analysts + IT administrators

Contact Information

If you have further questions or comments:

Wayne Eckerson, TDWIweckerson@tdwi.org

This presentation is based on the TDWI Best

Practices report “Beyond Reporting: Delivering

Insights with Next-Generation Analytics,” available for

download at www.tdwi.org/research/reportseries