BI in the Clouds (Wlodek Bielski Technology Stream)

BI IN THE CLOUDS

Wlodek Bielski BI Architect www.itmagination.com

Consultants Capacity (MH/Miesiąc)

Delivered projects Revenue (mUSD)

176 2014

280 2015

500 2016

29 568 2014

47 040 2015

84 000 2016

90 2013

174 2014

376 2015

6,1 2014

11,1 2015

20,5 2016

FACTS ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

SOFTWARE DEVELOPMENT

CLOUD OUTSOURCING MANAGED SERVICES

Dedicated software production

Application integration

Application migration

Application maintenance

Managing the process of software

development

.NET, ASP.NET MVC, TFS

Transition to the Cloud

Microsoft Azure

SharePoint Online

Office 365

Cloud Managed Service

Outsourcing IT professionals

Time & Material accounting model

Nearshoring / Offshoring

Transparent business model 80/20

Application Managed Services

Infrastructure Managed Services

Help Desk and Maintenance Managed Services

Nearshoring / Offshoring

PERFORMANCE MANAGEMENT

Budgeting, planning and forecasting

Financial Consolidation

Cash flow Management

Production Controlling

Integrated Reporting

Predefined applications: Solvency II, COREP,

FINREP

Exclusive distributor of Tagetik in Poland

DATA MANAGEMENT

Analytical - reporting solutions

Big Data

Data Integration

Data Warehousing

Business Analytics

Data Mining

Mobile BI Analytics

Digital Marketing Analytics

GLOBAL DELIVERY ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

Onshore

Offshore

International contracts

AGENDA ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

What is Cloud BI?

Key Drivers of Cloud BI

Cloud BI Delivery Models

Data Lake vs Data Warehouse

Case Study

Conclusions

WHAT IS BI? ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

BI

Data Integration

Data Warehouse

Analytics

Data Viz

Big Data

Data Science

KEY DRIVERS OF CLOUD BI ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

Speed of implementation and deployment Instant Dev and Test environments

Elasticity – scaling up/down instantly Focus on core capabilities – outsource if needed

More affordable Enhanced self-service capabilities Available anytime, anywhere

Lower TCO Less commitments

Cloud-born data

CLOUD DELIVERY MODELS ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - ■

On Premises

Storage

Servers

Networking

OS

Server Software

Data

Applications

You

man

age

IaaS

Storage

Servers

Networking

OS

Server Software

Data

Applications

You

man

age

Managed by Cloud

PaaS

Storage

Servers

Networking

OS

Server Software

Data

Applications

You

man

age

Managed by Cloud

SaaS

Storage

Servers

Networking

OS

Server Software

Data

Applications

You

man

age

Managed by Cloud

CLOUD DELIVERY MODEL: AWS ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

On Premises

Storage

Servers

Networking

OS

Server Software

Data

Applications

IaaS

S3

EC2

VPC

Linux

MySQL

Databases

Applications

PaaS

S3

EC2

VPC

Managed

RDS / Redshift

Databases

Applications

SaaS

Managed

Managed

Managed

Managed

Managed

Managed

QuickSight

CLOUD DELIVERY MODEL: AZURE ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

On Premises

Storage

Servers

Networking

OS

Server Software

Data

Applications

IaaS

Azure Storage

Azure VM

Azure VNet

Win Serv 2016

SQL 2016 Ent

Databases

Applications

PaaS

Managed

Managed

Managed

Managed

SQL DB / DWH

Databases

Applications

SaaS

Managed

Managed

Managed

Managed

Managed

Managed

Power BI

ON-PREMISES ETL PROCESS ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

Extraction Transformation Loading

Physical or virtual servers Software licenses cost Close network proximity Limited scalability

CLOUD ETL: IAAS ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

Extraction Transformation Loading

Virtual instances Software licenses cost / Pay-as-you-Go Proper network topology / VPN Security and Compliance

CLOUD ETL: PAAS ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

Typically ELT, not ETL SOA-like Orchestration

Extraction Loading Transformation

Great for cloud-born data Flexibility and scalability

Less moving parts Less control

Managed services Pay-As-You-Go High distribution Data Security

CLOUD ETL: DATA LAKE ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

Typically ELT, not ETL SOA-like Orchestration

Extraction Loading Transformation

Great for cloud-born data Flexibility and scalability

Less moving parts Less control

Managed services Pay-As-You-Go High distribution Data Security

DATA LAKE VS DATA WAREHOUSE ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

DATA WAREHOUSE DATA LAKE

Stores raw data in its native format

Supports multiple data formats

Schema-on-Read

Complex processing

Batch processing of data at scale

Stores cleansed and transformed data

Optimized for structured, relational data

Schema-on-Write

Complex queries

Thousands of users

Clickstream and logs

Sensor data

Images

Transactions

CLOUD ETL: HYBRID ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

Best of both worlds Leverages existing investments

Future-proof Evolves as Cloud matures

CASE STUDY

CUSTOMER’S PROFILE ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

Core platform – mySQL

Own traffic – REST API/JSON

Competitors traffic – REST API/JSON

Clickstream data - CSV/JSON

Internal systems

Global E-Commerce platform (C2C, B2C)

Both Horizontals and Verticals

Strong focus on services monetization

BUSINESS DATA

SYSTEMS

Legacy on-premises BI

Predefined reports and dashboards

PHASE 1: IAAS ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

SQL Server 2016 / Integration Services / Analysis Services

IaaS in Microsoft Azure AWS also considered Set of independent ETL processes

MySQL

REST API

Excel / Web

PHASE 2: HYBRID ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

SQL Server 2016 / Integration Services / Analysis Services

PaaS services Data Lake Data Warehouse Advanced Analytics

MySQL

REST API

Excel / Web

Data Lake

Data Factory

R / U-SQL

CONCERNS ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

Data Security and Compliance

Performance and networking issues

Complicated pricing models

Integration with on-premises solutions

Lack of control

CONCLUSIONS ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■

Cloud BI can complement traditional on-premises BI

Unlocks potential in all types of data, anywhere, anytime

Production-ready, but needs to mature in some areas

First choice for Cloud-born data

Allows for true democratization of BI

... More fun to come!

THANK YOU!

Wlodek Bielski Business Intelligence Architect

wlodzimierz.bielski@itmagination.com

CONTACT ■ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ■