DBI405. Agenda Reporting Services Scale Out Architecture Report Catalog Best Practices Scale Out Deployment Best Practices Performance Optimization.

Scale-Out Deployment of Microsoft SQL Server Reporting Services

Carl Rabeler, Prash ShirolkarProgram ManagersMicrosoft

DBI405

Session Objectives And Takeaways

Session Objective(s): Provide guidance on how to scale out your Reporting Services environment Provide RS best practices on RS catalogs, scale out deployment, and performance optimizations

Reporting Services Scale Out Architecture

Agenda• Reporting Services Scale Out Architecture• Report Catalog Best Practices• Scale Out Deployment Best Practices• Performance Optimization Configurations• Disaster Recovery and Scale-Out• Troubleshooting Tips

Report Catalog

Reporting Data

Scaling Out Architecture: Native ModeOverall Architecture

RS S

cale

Out

Dep

loym

ent

RSDB

Clients

Flat Files,OLE DB,ODBC

SQL, AS,DB2, Oracle,

Teradata, etc.

RS Server

RS Server

RS Server

Report Server

NLB

Clients

Clients

Data Flow in Native Mode

1. Report request from user to Reporting Services engine2. Reporting Services engine retrieval of report definition from RS catalog server3. Reporting Services engine query request to data source4. Return of the rendered report from the Reporting Services engine to the user

Shar

ePoi

nt W

eb F

ront

End

Ser

vers

Catalogs

Reporting Data

Scaling Out Architecture: SharePoint ModeOverall Architecture – Pre Denali

RS A

pplic

ation

Ser

vers

RSDB

Clients

Flat Files,OLE DB,ODBC

SQL, AS,DB2, Oracle,

Teradata, etc.

RS App Srvr

NLB

Clients

Clients

WFE

RS App Srvr

RS App Srvr

WFE

WFE

SharePoint Farm

Content DB

NLB

Data Flow in SharePoint Mode

1. Report request from user to the SharePoint web service2. SharePoint engine retrieval of the report definition from the SharePoint catalog (the master copy)3. SharePoint web service report request to Reporting Services engine4. Reporting Services engine query to the RS catalog server to verify that the report definition exists in the RS catalog and that it is same as the master copy5. Reporting Services engine query request to data source6. Reporting Services forward of rendered report to the SharePoint web service7. Return of the report from the SharePoint web service to the user

Scale Out Architecture Performance Illustration

1 server 2 servers 4 servers0

500

1000

1500

2000

2500

608

1218

2300

Max # of Concurrent Users

Users

Overview of Scale Out (and Scale Up)

Scale up RS front-end servers for performance (e.g. four-processor, quad-core) Scale out to multiple RS front end servers for performance and high availabilityUse dedicated RS Catalog servers - scale up for performance and cluster for availability In SharePoint mode, use dedicated RS application server(s) and scale up and/or scale out WFE serversOptimize disk I/O subsystem for maximum performanceOptimize network links to ensure sufficient bandwidth

Scale Out ArchitectureRead the manuals!

Read these SQL CAT papers:Reporting Services Scale-Out Architecturehttp://sqlcat.com/technicalnotes/archive/2008/06/05/reporting-services-scale-out-architecture.aspx

Report Catalog Best Practiceshttp://sqlcat.com/technicalnotes/archive/2008/06/26/report-server-catalog-best-practices.aspx

Reporting Services Scale-Out Deployment Best Practiceshttp://sqlcat.com/technicalnotes/archive/2008/10/21/reporting-services-scale-out-deployment-best-practices.aspx

Reporting Services Performance Optimizationshttp://sqlcat.com/technicalnotes/archive/2009/01/14/reporting-services-performance-optimizations.aspx

SQL Server Reporting Services Disaster Recovery Case Studyhttp://sqlcat.com/technicalnotes/archive/2009/10/23/sql-server-reporting-services-disaster-recovery-case-study.aspx

Read these TechNet ArticlesConfiguring Reporting Services for Scale-Out Deploymenthttp://technet.microsoft.com/en-us/library/ms156453.aspx

How to: Configure a Report Server Scale-Out Deployment (Reporting Services Configuration)http://technet.microsoft.com/en-us/library/ms159114.aspx

Deployment Topologies for Reporting Services in SharePoint Integrated Modehttp://technet.microsoft.com/en-us/library/bb510781.aspx

Scale Out ArchitectureImportance of Performance Testing

Need to understand your scenarios and reportsScenarios are defined by user personas & usage patternsReports are either test reports or actual reports Tests should isolate Report Server from other systems

Need tools to automate the testingSee white paper: Using Visual Studio 2005 to Perform Load Testing on a SQL Server 2005 Reporting Services Report ServerMake single incremental changes between testsDo not use SQL trace inside VSTE

Report CatalogBest Practices

Agenda• Reporting Services Scale Out Architecture• Report Catalog Best Practices• Scale Out Deployment Best Practices• Performance Optimization Configurations• Disaster Recovery and Scale-Out• Troubleshooting Tips

Report Catalog

Reporting Data

Report Catalog Best Practices

RS S

cale

Out

Dep

loym

ent

RSDB

Clients

Flat Files,OLE DB,ODBC

SQL, AS,DB2, Oracle,

Teradata, etc.

RS Server

RS Server

RS Server

Report Server

NLB

Clients

Clients

Native Mode

Report Catalog Best PracticesReport Server Catalog Breakdown

Report Server Catalog (RSDB)Stores all report metadata including report definitions, report / history snapshots, scheduling, etc.

RS Temp DBStores temporary snapshots while running reports

These databases can be a bottleneckOptimize by applying standard SQL DB techniquesCatalog has a lot of I/O and transactions

RS2005: Many inserts to ChunkData, SnapshotData, and SessionData tablesRS2008 and RS 2008R2: Many inserts Segment; takes majority of transactions of RSTempDB

Repo

rt C

atal

og

RSDB

Report Catalog Best PracticesUse a dedicated catalog server

Common (not best performance) scenariosSame server as SSRS Server

Great for small environmentsIn enterprise environments, too much resource contention

Same server as data source databaseSQL resource contention (TempDB, plan cache, memory buffer pool) between data source and RS catalogsAs load increases need to monitor CPU, I/O, network resources, and buffer pool

For best performanceReduce resource contention by having a dedicated RS catalog server you can tune

RSDB

Shar

ePoi

nt W

eb F

ront

End

Ser

vers

Catalogs

Reporting Data

Report Catalog Best PracticesSharePoint Mode

RS A

pplic

ation

Ser

vers

RSDB

Clients

Flat Files,OLE DB,ODBC

SQL, AS,DB2, Oracle,

Teradata, etc.

RS App Srvr

NLB

Clients

Clients

WFE

RS App Srvr

RS App Srvr

WFE

WFE

SharePoint Farm

Content DB

NLB

Report Catalog Best PracticesHigh Performance Disk

Check out Predeployment I/O Best PracticesHave more smaller size disks with faster rotation speeds (>=15k RPM) vs. fewer larger disks with slower rotationsMaximize/balance I/O across ALL available spindlesSeparate disks between RSDB and RSTempDB

RSDB a lot of small transactions (report metadata)RSTempDB has more (not as many) larger transactions

In SharePoint mode, use separate disks from the content DBsPre-grow your databasesStripe dB files to number of cores (0.25 – 1.0)

Minimize allocation contentionEasier to rebalance database when new LUNs are available

Use RAID 10, not RAID 5

RSDB

Report Catalog Best PracticesOperations Best Practices

Data in RSTempDB is highly volatileReport lifetime policy of data = SessionTimeout value (10min)CleanupCycleMinutes guides background cleanup threadOnce session timeout reached, cleanup temporary snapshot from tempDBThis is done every CleanupCycleMinutes

Data is RSDB is long lived; should be backed upBacking Up and Restore Databases in SQL ServerOptimizing Backup and Restore Performance in SQL ServerBacking Up and Restore Encryption KeysSharePoint backup does not back up the RSDB – until Denali

Maintain your RS catalogsRemember, these are SQL databasesE.g. Re-indexing catalog tables or updating stats may improve query performance

RSDB

Scale Out DeploymentBest Practices

Agenda• Reporting Services Scale Out Architecture• Report Catalog Best Practices• Scale Out Deployment Best Practices• Performance Optimization Configurations• Disaster Recovery and Scale-Out • Troubleshooting Tips

Report Catalog

Reporting Data

Scale Out Deployment BP

RS S

cale

Out

Dep

loym

ent

RSDB

Clients

Flat Files,OLE DB,ODBC

SQL, AS,DB2, Oracle,

Teradata, etc.

RS Server

RS Server

RS Server

Report Server

NLB

Clients

Clients

Scale Out Deployment BP Network Load balancing important for many client

connections to RS servers

Recommend: Use cookie persistence to preserve SSRS-to-client connection

IP affinity can work but may be overload for browser-based connectionsKeep round-robin for initial connections

Recommend: dual NIC for RSSplit browser and RS/DB traffic

RS S

cale

Out

Dep

loym

ent

Clients

RS Server

RS Server

RS Server

NLB

Clients

Clients

Shar

ePoi

nt W

eb F

ront

End

Ser

vers

Scaling Out Architecture: SharePoint ModeNetwork

RS A

pplic

ation

Ser

vers

Clients

RS App Srvr

NLB

Clients

Clients

WFE

RS App Srvr

RS App Srvr

WFE

WFE

Load balancing important for many client connections to RS serversBalancing by WFE across multiple RS application servers handled by SharePoint

Recommend: dual NIC for RSSplit browser and AS/DB trafficNLB

Scale Out Deployment BPReport Data Performance Considerations

Scale out works for RS but may not work (or may require different techniques) for underlying Report Data (data source)

Reporting loads Report Data, \ limit impact of large numbers of users

Limit data set size using report filtersSSIS to limit data from Operational data sourcesDo not let all users access all of the reportsUse shared data sets to enable DBAs to optimize query performance without having to be Reporting Service experts

Reporting Data

Flat Files,OLE DB,ODBC

SQL, AS,DB2, Oracle,

Teradata, etc.

Performance Optimization Configurations

Agenda• Reporting Services Scale Out Architecture• Report Catalog Best Practices• Scale Out Deployment Best Practices• Performance Optimization Configurations• Disaster Recovery and Scale-Out • Troubleshooting Tips

Performance OptimizationUse 64-bit hardware and software

RS Catalog• It definitely helps the RS Catalog SQL servers

Report Servers• RS requires each data set to fit in memory

E.g. 32-bit RS will hit 3GB ceiling IIS process recycles = reports fails

• 64-bit does not increase the speed of report rendering• Allow users to view and export larger reports• May get better throughput at higher workloads due to less

memory contention

SharePoint application servers handle more concurrency with more memory – SharePoint 2010 is 64-bit only

RS Server

Performance OptimizationFeatures

RecommendationsUse Report Execution Timeouts to control how a long a report has to execute

For large reports with data processing bottlenecks, use Scheduled Snapshots

Rendering reports into non-browser formats like PDF and Excel, address this by scheduling Delivered Rendered reports

Pre-populate the report cache using data driven subscriptions

Remember, RS is designed for interactive use; If you want to export data, use file share delivery or SSIS as your mechanisms

RS Server

Performance OptimizationConfigurations

Recommendation: • Disable report history to reduce/remove the report history

snapshots• If history desired, establish global or report-specific limit

to the number of snapshots

Recommendation: • Understand the need for big reports• Consider Cache and Snapshot execution options• Mitigates against load cause by concurrent users

RS Server

Performance OptimizationMemory Configurations

MemoryLimit Configuration• Default 60% of available memory• Increase help process more requests• Once threshold hit, no new requests are accepted

MaximumMemoryLimit Configuration• Default 80% of available memory• If this threshold is met, processing is aborted

Changing values may solve RS only to bring up other contentions

Recommendation: If constantly hitting memory thresholds, consider scaling up and then scaling out

RS Server

Disaster Recovery and Scale-Out

Agenda• Reporting Services Scale Out Architecture• Report Catalog Best Practices• Scale Out Deployment Best Practices• Performance Optimization Configurations• Disaster Recovery and Scale-Out • Troubleshooting Tips

Basic Disaster Recovery Scenario

Primary Data CenterUse failover cluster for RS catalogsUse asynchronous mirroring to synch RS database with DR site

DR Site Closely duplicate primary data centerGeographically separatedFiber connectivity between sites to minimize latencyIdentically named instances

Content switchTo load balance traffic within site and between global sites (when needed)

Reporting Services Servers within the Primary Data Center Go Offline

Auto failover to RS Servers in DR Site

Content switch detects outage and will automatically direct traffic to RS Servers in DR site

Continue reading source data from primary site

The primary site has the most up-to-date data

Client connectivity to RS environment is minimally impacted

Active Cluster for RSDB in the Primary Data Center Goes Offline

Auto failover to secondary in failover cluster

Clustering services will detect failure and automatically fail over

Continue reading source data from promoted secondary in primary data centerClient connectivity to RS environment is minimally impacted

Entire Cluster for RSDB in the Primary Data Center Goes Offline

Manual failover to RS Servers in DR Site

RS servers must be manually configured to point to the RSDB in the DR siteSimplified by identical instance names

The DR Site may not have the most recent data due to asynchronous mirroring

May be some data loss, but it should be minimal as RSDB data not particularly volatile

Client connectivity to RS environment is minimally impacted

Reporting Services Troubleshooting Tips

Agenda• Reporting Services Scale Out Architecture• Report Catalog Best Practices• Scale Out Deployment Best Practices• Performance Optimization Configurations• Troubleshooting Tips

Troubleshooting TipsOverview

Reports can take up a lot of memory a lot of time to execute a lot of CPU

It is possible for errors to occur as a result: Out of MemoryInternal ErrorsRendering errors

So how do we diagnose issues when running reports in the Report Server?

TroubleshootingStart with the RS Logs Files

Reporting Services execution logContains data about specific reports and their performanceStored in Report Server database

Report Server service trace logDetailed error stacks showing what the problems areLocated under:

<install folder>\Reporting Services\LogFiles

Windows Application logMajor events are in the Windows application event log

ReferencesReporting Services Log Files details are in BOL at: http://msdn2.microsoft.com/en-us/library/ms157403.aspx RS Error code info:http://msdn2.microsoft.com/en-us/library/ms165307.aspx

Troubleshooting TipsMonitoring Performance

Want to look at memory consumptions, application domain recycles, cpu usage, etc.

Good resources for this:This topic describes the performance countershttp://msdn2.microsoft.com/en-us/library/aa972240(SQL.80).aspx

Analyzing and Optimizing Reports – ExecutionLog2 Viewhttp://blogs.msdn.com/b/robertbruckner/archive/2009/01/05/executionlog2-view.aspx

Monitoring Report Execution Performance with Execution Logs http://msdn2.microsoft.com/en-us/library/aa964131.aspx

Monitoring interactive report executions via ListJobs SOAP API to see which long running reports are currently executinghttp://msdn2.microsoft.com/en-gb/library/aa225969(SQL.80).aspx

Use the SQL Server Profiler to monitor actions of the Report Server database

Troubleshooting TipsUsing ExecutionLog2

Syntaxuse ReportServerselect * from ExecutionLog2 order by TimeStart DESC

Use to answer questions such as: What are my long running reports, and where is the time spent for these reports?Which reports might be good candidates for caching?How many reports were returned from cache vs. live execution vs. execution snapshot?What was the most popular report for the week?What are the poor performing reports that need to be taken action upon?Which reports have large outputs, such as PDF?Which reports utilize a large amount of memory?

Troubleshooting TipsDiagnosing Problems

Specific actions to help diagnose problems:Adjust memory limits:• Refer to “Report Size in Memory” section of

http://msdn2.microsoft.com/en-US/library/ms156002.aspx

Adjust schedule concurrency:• Do this to isolate which are your problem reports• May want to reduce number of simultaneous report executions to 1

especially for extremely large reports• In rsreportserver.config, set <MaxQueueThreads>

• Zero (0) means RS server automatically determines the “right number”

Troubleshooting TipsCommon problems

Error 401 is an authentication error Providing no or incorrect credentialsAlso can be seen during an ASP.NET application reset as wellHappens if server starts running out of memoryCheck logs for InternalCatalogException;

in the stack trace, you should see the method that failed

Within trace logs - Application Domain RecyclesApp domain recycles used to clear out memoryOften will be correlated with interactive report execution failuresIndicates RS server is under memory pressure

Troubleshooting TipsCommon problems (Continued)

SQL Query Plan optimizationStored Procedures require query plan optimizationUse the With Recompile directive in Stored ProceduresUse fully qualified names to access across DB objects

Refresh on SQL Query Performance:http://msdn2.microsoft.com/en-us/library/ms187032.aspx

Report Design with drop-down parameter listsEvaluating drop downs results in query load on DBMSOptimize parameter, prompts, and valid values

What’s Next?

Follow, Tweet and Enter to win an Xbox Kinect Bundle

GAME ON! Join us at the top of every hour at the BI booth to compete in the Crescent Puzzle Challenge and Win Prizes

Sign up to be notified when the next CTP is available at: microsoft.com/sqlserver

@MicrosoftBI

/MicrosoftBI

Join the Conversation

Resources

www.microsoft.com/teched

Sessions On-Demand & Community Microsoft Certification & Training Resources

Resources for IT Professionals Resources for Developers

www.microsoft.com/learning

http://microsoft.com/technet http://microsoft.com/msdn

Learning

http://northamerica.msteched.com

Connect. Share. Discuss.

Complete an evaluation on CommNet and enter to win!

Scan the Tag to evaluate this session now on myTech•Ed Mobile

© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS

PRESENTATION.