ManagedSelfServiceBI_MelissaCoates_v5

Managed Self-Service BI

&

Data As A Service

PASS DW/BI

Virtual Chapter

7/12/2012

Melissa Coates

Melissa Coates

Blog: http://www.sqlchick.com

Twitter: @sqlchick

About Melissa • Business Intelligence developer

based in Charlotte, NC

• Sr. Consultant with Intellinet

• Specialize in BI and Data

Warehousing solutions using the

Microsoft platform

About Intellinet

Strategy Business

Cost

Reduction

Cycle Time Reduction

Supply Chain, Systems,

& Software Optimization

Business Productivity

Solutions

Revenue

Enablement

E-Commerce Solutions

Commercial Software

Development

Merger & Acquisition

Assimilation

Process

IT Strategy & Operations

IT Strategy Roadmaps &

Benchmarks

Assessment, Planning, &

Governance

Portfolio Management

Continuous Improvement

Programs

Business Process

Optimization

Project & Program

Management

Service Desk &

IT Operations

Business Analysis

& Quality Control

Change Management

Technology

Portals &

Collaboration

Intranets & Extranets SharePoint

Cloud-based Solutions Office 365 / Azure

Corporate Social

Media Solutions Yammer / Twitter

Facebook / LinkedIn

Business

Intelligence

Data Warehouses,

Reports & Analytics

Front-end Tools

PerformancePoint / Excel

Data Integration &

Management SQL / SSIS / SSAS / SSRS

Application Development

Custom Development .NET

Application Lifecycle

Management Visual Studio / TFS

Business Process &

Integration BizTalk / SOA

Infrastructure

Server Platform Virtualization Cloud Computing

AD / Exchange / Windows Server Hyper-V Office 365

Intellinet is a management consulting

and

Microsoft-centric technology services firm.

http://www.intellinet.com

Agenda • Introduction to “Managed” Self-Service BI

• Overview of Microsoft Self-Service components

Demo: PowerPivot | Power View | PowerPivot Gallery

• Techniques to Monitor, Secure, & Manage SSBI

environment

Demo: PowerPivot Management Dashboard

• Introduction to Data As A Service (DaaS)

• Wrap-up: Keys to Success with SSBI

Not in scope for our discussion: Installation & configuration of

PowerPivot for SharePoint. See MSDN + this TechEd 2012 recording:

http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/DBI402

INTRODUCTION

TO

“MANAGED”

SELF-SERVICE BI

Self-Service BI – what do we really mean? Commonly thought of:

“Empower users to create their own reports so users get

what they want without having to ask IT.”

Generally 2 groups of users:

Data Analysts

(Power Users)

True ad-hoc needs

Direct data access

Small % of users

Producers of data

Data Consumers

(Casual Users)

Guided ad-hoc needs

Parameterized reports

Big % of users

Consumers of data

Corporate BI

IT pro

Source Data

ETL

Data Warehouse,

Cubes Corporate

Reports,

Dashboards,

Scorecards

Business

Users

Corporate BI + Self-Service BI

IT pro

Source Data

ETL

Data Warehouse,

Cubes Corporate

Reports,

Dashboards,

Scorecards

Business

Users

Data Feeds,

PowerPivot Models,

Excel Services,

Misc Files

Data Feeds,

PowerPivot

Models,

Misc DBs,

Misc Files

A single managed

environment (SharePoint Portal)

Business

Users

Business Reports,

Dashboards, Scorecards,

PowerPivot Models

Producers

Consumers

Needs driving SSBI 1. Productivity / time

• Long dev cycles for IT to develop ETL, DW, OLAP

• IT backlog of requests

• Business decision may need to be made quickly with

whatever information is readily available

2. Data exploration / unpredictable ad-hoc analysis

• Requirements aren’t always known or predictable

• A one-time analysis may not justify augmenting the

existing BI solution

3. Prototyping for a Corporate BI solution

• Convey requirements

• Reduce development cycles

Challenges with SSBI 1. Training of producers on data & tools

• Need to understand the data and how to use the tools

• Recognition when SSBI works vs. Corporate BI

2. IT support

• Some IT staff have the “users can’t handle it” syndrome

• Instead of an IT backlog for reports; SSBI creates different

demands: training, support help, documentation

3. Access of questionable data sources

4. Chaotic, unorganized environment

5. Lack of testing & validation

6. Lack of governance & change management

“Managed” Self-Service BI

Managed, Monitored, Secured by:

IT Staff

Power View

Report Builder

Delivery:

SharePoint 2010 Portal

Producer:

Data Analysts -or- IT

Producer:

IT

PerformancePoint

Reporting Services

Visio Services

Self-Service BI Tools

Corporate BI Tools

PowerPivot

Excel 2010 with

Excel Services

OVERVIEW

OF MICROSOFT

SELF-SERVICE BI TOOLS

PowerPivot • Add-in to Excel 2010

• In-memory solution

• Based on xVelocity (Vertipaq)

column-store indexes

• Large volumes of data

• Create “mashups” of data

• Data is embedded

• Introduces DAX

• Schedule data refreshes in

SharePoint

• Can do visualization in familiar

Excel environment, or another tool

PowerPivot for SharePoint

PowerPivot for Excel

Excel with Excel Services

• Displays workbook

on Web

• Share an entire

workbook or

sections

• Not all Excel

functionality is

supported

• Integrated with

PowerPivot for

SharePoint

Power View

• Visual, interactive

reporting

• Unique data

discovery

• Presentation-ready

(like PPT slides)

• Silverlight-based

• Requires a Tabular

source (either

PowerPivot for

SharePoint or a

Tabular SSAS model)

• Requires Reporting

Services Add-In

Report Builder

• ClickOnce application

• Pixel-perfect, fully

formatted reports

• Export capabilities

• Subscription delivery

• Extremely powerful tool if

report developer is clever

with expressions (Visual

Basic)

PowerPivot Gallery

• Specialized

SharePoint

document

library

• Thumbnail

previews

• Manage

data

refreshes for

PowerPivot

workbooks

Demo Create Model with PowerPivot

Publish Model to PowerPivot Gallery

Visualize Data with Power View

TECHNIQUES TO

MONITOR, SECURE, &

MANAGE THE

SELF-SERVICE BI

ENVIRONMENT

PowerPivot for SharePoint

Source: http://msdn.microsoft.com/en-us/library/ee210692.aspx

xVelocity Engine

& Data Cache

SSAS Instance for PowerPivot If displayed

in SSMS:

The data is

actively

loaded to

memory

• User uploads workbook Added to Content DB

• Query issued Data is loaded into memory

Kept in memory for 48 hours (if no other memory pressure) at which

time data is unloaded to the cache (kept for another 72 hours in

cache). If 5 days pass with no activity, cache file is physically

deleted. Cache files stored: \Microsoft SQL Server\MSAS11.PowerPivot\OLAP\Backup

Data Refreshes – 2 Approaches

PowerPivot for Excel: manually refreshed

PowerPivot for SharePoint:

1. User refreshes while the workbook is open. **Does not refresh data in the PowerPivot database.**

2. User sets up scheduled data refreshes. This method does refresh both the Excel workbook +

the PowerPivot database. (It actually sets the

“Refresh on Open” flag in the Excel workbook to make

the Excel file update itself when user opens it.)

Managing PowerPivot in SharePoint Workload on

the server

from the

PowerPivot

models

• Monitor server health incl. query durations

• Teach authors not to retrieve every field in

model (it’s all in memory after all!)

• Manage memory availability: all db’s in use

must be in memory concurrently (leaving a

~10%-20% buffer)

Data Refresh

Schedule

• Monitor refresh times & durations

• Ensure users must use the “data refresh

account configured by the administrator” & not

their own credentials

• Users have individual access to misc db’s

Disk space • Monitor disk space (files are cached to avoid

round trips to content db)

1/2

Managing PowerPivot in SharePoint Naming &

Versioning of

PowerPivot

Models

• Teach publishers to continue using same

name (i.e., avoid habit of V1, V2, or dates at

end of file names)

• Consider minimal versioning in library (content

db size issues since data is embedded in the

workbook)

• Monitor file uploads for storage requirements

(since data is embedded)

Information

about the

Model

• Requiring certain metadata fields for reports &

models will aid with Search (can’t see custom

fields in Gallery view though)

2/2

Securing the PowerPivot Gallery Permissions to

publish

• Consider using workflows for approval of

new models being published

Access to

PowerPivot

models

(aka the

“workbook-as-a-

data-source”)

• Access to the PowerPivot Gallery

document library

• Separate SPS document libraries

• Based on content

• Based on security

• Permissions on individual models

Access to reports

(Power View,

Excel Services,

Report Builder)

Same as above plus

• Permissions on reports are preferably the

same as the underlying model

1/2

Securing the PowerPivot Gallery Thumbnail

previews –

possibility of

showing a

preview for data

the user cannot

see

• Set permissions on report the same as

the model (possible security hole)

• Use a regular document library (instead

of PowerPivot Gallery) if preview is of

concern

Access to data

contained within

the PowerPivot

workbook

• View Only permissions: user will get a

“snapshot” only; no data is exposed

• Contribute permissions: user can

download full workbook & access all data

stored within the PowerPivot model

2/2

Model

Library

Report

Managing PowerPivot on Desktops RAM • RAM upgrades may be needed on user

machines (min 4GB – give 6GB or 8GB if

possible)

64-bit vs. 32-

bit version

• 32-bit version cannot handle data volumes

>1M rows

• 64-bit version not compatible with all Office

add-ins (although the Data Mining add-in is

now supported with the latest update)

Query syntax

& dynamic

data

refreshes

• IT staff may need to initially develop

PowerPivot models & have business users

maintain them (a pretty common scenario

currently)

Reporting on PowerPivot Usage

SQL Server database

PowerPivot Service

Application

SSAS

Tabular Model

Management

Data Sandbox

PowerPivot

Management

Dashboard

PowerPivot Management Data:

Ad-hoc

reporting

Customizable

dashboard

(need Central

Admin

access)

PowerPivot Management Dashboard

PowerPivot Management - Reports

Visibility into: • Queries

• CPU

• Memory

• Connections

• Workbook Sizes

• Users & Authors

• Usage Increase & Decrease

• Data Refresh

PowerPivot Management - Refreshing

http://msdn.microsoft.com/en-us/library/ee210657.aspx

Relies on 3

timer jobs:

Reporting on PowerPivot Usage

1

2

3

4

Demo

PowerPivot Management Dashboard

Ad-Hoc Reporting on PowerPivot Management Data

Managing Report Builder Reuse &

centralization

• Shared Data Sources (connection string)

• Shared Datasets (queries)

• Report Parts (reusable objects like charts,

tables, parameters)

Semantic

layer

• Database views

• Cube perspectives

• Report Models are deprecated

Deployment

location

• Separate SPS document libraries

• Based on content

• Based on security

Workload on

server

• Global report timeout setting

1/2

Managing Report Builder Starter reports • Deploy “starter reports” with connections

already in place

Logging • Enable Execution Logging (default: 60 days

– probably want to extract & store

historically)

Printing • Enable Client Printing - An ActiveX control is

required for the print button on the toolbar

2/2

Managing Excel Services Trusted locations • Trusted Data Connection Library

• Trusted File Locations

• Trusted Data Providers

Office Data

Connection

(ODC) files

• House within a Data Connection Library

• Use consistent, friendly names

• Consider linked connections, instead of

embedded

Starter reports • Deploy “starter reports” with connections

already in place (helps too if users are

PivotTable-challenged)

Workload on

server

• Data connection timeouts

• Allocation of memory

Securing Excel Services Access to data

contained within

the Excel

Services

workbook

• View Only permissions: user can open,

interact, refresh workbook; a “snapshot”

may be downloaded; no data is

exposed same as PowerPivot

• Read permissions: user can download

full workbook & access all data published

via Excel Services different than

PowerPivot

• Contribute permissions: in addition to

Read, user can update & delete

• Design permissions: in addition to

Contribute, user can approve

Managing Power View Silverlight • Requires Silverlight 5

Data source

required

• Requires a tabular model

• PowerPivot, or

• Analysis Services in Tabular Mode

BI Semantic

Model (BISM)

connection files

• PowerPivot for SharePoint

• HTTP link to model, or

• BISM data connection

• Analysis Services Tabular Model

• BISM data connection

Although BISM files are similar to ODC files,

they don’t have the same trust settings

required by Excel Services

INTRODUCTION TO

“DATA AS A SERVICE”

IN SUPPORT OF

SELF-SERVICE BI

Why DaaS? • You might have created a centralized report catalog

– but how about a centralized data catalog?

• A centralized data abstraction layer allows users to

explore and consume data (and perhaps publish)

• Microsoft examples of DaaS: • Windows Azure Marketplace (DataMarket)

• An internal Silverlight-based application called DSL (Data Services

Layer) as part of their CBI (Consolidated BI) environment

• Data Feeds library

• SQL Azure Labs Data Hub (not a released product yet)

Corporate BI + Self-Service BI + DaaS

IT pro

Source Data

ETL

Data Warehouse,

Cubes Corporate

Reports,

Dashboards,

Scorecards

Business

Users

Data Feeds,

PowerPivot Models,

Excel Services,

Misc Files

Data Feeds,

PowerPivot

Models,

Misc DBs,

Misc Files

“Enterprise

Report Catalog” (SharePoint)

Business

Users

Business Reports,

Dashboards, Scorecards,

PowerPivot Models

Power Users

Consumers

Centralized

Data

Catalog

How Microsoft does DaaS 1/2

Source: http://technet.microsoft.com/en-us/library/hh499043.aspx

How Microsoft does DaaS

Source: http://technet.microsoft.com/en-us/library/hh499043.aspx

Data Services Layer: SQL APIs,

Web Services, and UI controls

2/2

Centralized Data Catalog • Consider including:

• Description of data

• Business justification

• Owner name

• Security model (i.e., unsecured or limited)

• Availability dates (i.e., if limited time)

SQL statements

Version info re: changes

Sample data

Data Explorer A self-service tool to Discover, Enrich & Publish data

• Web-based client is “Cloud Service”

preview (SQL Azure Labs)

• Desktop client has an Excel add-in;

not as full-featured as Cloud

Data Hub A Windows Azure service to create & manage a private

data marketplace for your enterprise data.

In a “Cloud Service” preview (SQL

Azure Labs). In the 1st milestone

of the “Data Hub” roadmap.

WRAP-UP:

KEYS TO SUCCESS

WITH SELF-SERVICE BI

Keys to Success

• Work with strengths & limitations of Self-Service BI

• Find where it complements Corporate BI in your org

• Training & documentation

• User support

• Executive support & evangelists

• Governance

• Change management

Further Reading

Increasing Productivity by Empowering Business Users with Self-Serve BI

Choose the Right Business Intelligence Technology to Suit Your Style

Self Service Key to Creating Enterprise Business Intelligence Heroes

A Primer on PowerPivot Topologies and Configurations

Self-Service BI: Remember to Plan for the Back-End Infrastructure

Enabling Data as a Service for Self-Service Business Intelligence

PowerPivot Management Dashboard

TechEd 2012 Presentation: Deploying and Managing a PowerPivot for

SharePoint Infrastructure Using Microsoft SQL Server 2012

Thanks for attending!

Melissa Coates

Blog: http://www.sqlchick.com

Twitter: @sqlchick

Creative Commons License:

Attribution-NonCommercial-NoDerivative Works 3.0