How to Evaluate SAP BusinessObjects Out of Memory Generated Java Heap Files

8/9/2019 How to Evaluate SAP BusinessObjects Out of Memory Generated Java Heap Files

http://slidepdf.com/reader/full/how-to-evaluate-sap-businessobjects-out-of-memory-generated-java-heap-files 1/19

How to Evaluate SAP BusinessObjects Out ofMemory Generated Java Heap Files

Applies to:SAP BusinessObjects BI 4.0 services running under Adaptive Processing Server such as the DSL Bridgefulfilling query requests by for instance Web Intelligence using the BICS connectivity on BW.Working title: “How to evaluate SAP Busin essObjects out o f memo ry generated java heap f i les ”

Summary:

This paper provides a step by step description incl. screenshots to evaluate out of memory generated javaheap files and especially what was the root-cause for an out of memory issue. Understanding the reasons foran out of memory issues can increase the stability and user convenience of SAP BusinessObjects Enterpriseand tools such as Web Intelligence and other reporting tools that running under Adaptive Processing Server.The results of the tracing exercises can give important information on why an out of memory incidentoccurred and how to avoid this in the future. The result from the tracing exercise can for instance be used toredesign BEx Query for the purpose of consuming data in Web Intelligence reports.Without evaluating the cause of an out of memory incident it is almost impossible to discuss on how to avoidthis in the future. With this paper you should be able to find the exact root-cause of an out of memoryincident. With this paper you will get insights in how to trace out of memory java heap files yourself and tosome degree understand the inner workings of the Adaptive Processing server hosting the Web Intelligence.

Authors Bio

Kurt Holst ([email protected]) is an expert on SAP BusinessObjects Enterprisearchitecture, configuration and performance optimization. Kurt has worked with most majorOLAP databases, SAP NetWeaver BW and relational databases together with different BIreporting tools. Before joining SAP Kurt has worked as a BI Architect & developer bothinternal and as an external consultant. Furthermore Kurt has been attached to the universityas an external lector on Business Intelligence. Kurt is organizational with the Business

Analytics Services team in SAP Denmark but currently works out of the SAP Office inLevallois, France.

Company: SAPLast update: January 10 2013

mailto:[email protected]






How to Evaluate SAP BusinessObjects Out of Memory Generated Java Heap Files

2

TABLE OF CONTENTS

BACKGROUND AND THE NEED FOR EVALUATING JAVA HEAP FILES .................................................. 4

STEPS REQUIRED FOR EVALUATING THE JAVA HEAP FILE ................................................................... 7 Heap Dump generation: .................................................................................................................................. 7 Using Memory Analyzer Tool (MAT) to Analyze Java Heap Dumps: ......................................................... 8 Evaluating the java heap log file using memory analyzer ........................................................................... 8 Instability or performance issue – key findings and technical background: .......................................... 10

STEPS TO PREVENT OOM FROM HAPPENING AGAIN ............................................................................ 14

HOW TO INCREASE THE STABILITY OF SAP BO ENTERPRISE ............................................................. 15

REFERENCE DOCUMENTATION ................................................................................................................. 18






4

BACKGROUND AND THE NEED FOR EVALUATING JAVA HEAP FILES

The BusinessObjects Enterprise platform is a highly flexible solution that allows user with many differentreporting needs on place to access this information.

To make sure the users archive highest possible conveniences there is a need to proactively maintain andoptimize the BusinessObjects Enterprise services and when an incident occurs such as if the AdaptiveProcessing Server runs out of memory and generates a java heap file the root cause should be determinedto avoid this in the future. This article describes how to evaluate the java heap file and interprets theinformation to allow for highest possible stability and performance in your BusinessObjects Enterpriseenvironment.

The cause of less than optimum stability users experience with ad hoc reporting in SAP BusinessObjectsWeb Intelligence using BW as data sources with the BICS connectivity has in some cases been identified asrelated to design of the BEx Query.

The obvious question here is how stability with Web Intelligence on BW using BICS can be improved by re-designing on either Web Intelligence or the SAP BW side? This will also be discussed.

Figure 1: Overview of SAP BusinessObjects Enterprise Architecture

As shown on the overview of the SAP BusinessObjects Enterprise Architecture the APS is a part of theProcessing tier. The BI4.0 platform is by default installed with one Adaptive Processing Server (APS) perhost system. The Adaptive Processing Server is a generic server that hosts multiple services responsible forprocessing various requests. Services that can be hosted on this server include: Adaptive ConnectivityService, BEx Web Application Service, Client Auditing Proxy Service, Custom Data Access Service, Data

Federation Service, Document Recovery Service, DSL Bridge Service, Excel Data Access Service, LifecycleManagement ClearCase Service, Lifecycle Management Console Service, Monitoring Service, Multi-




5

Dimensional Analysis Service, Platform Search Service, Publishing Post Processing Service, PublishingService, Rebean Service, Security Token Service, Translation Service, Visual Difference Service,Visualization Service, Web Intelligence Monitoring Service.

Definitions:

“BICS" is short for Business Intelligence Consumer Services which is the connectivity allowing SAP

BusinessObjects tools such as Web Intelligence or Dashboard Designer.

”DSL” or The Dimensional Semantic Layer Bridge is what actually connects to BW. It is hosted by the

Adaptive Processing Server. The Adaptive Server hosting the DSL Bridge should be sized for expected user

load. The DSL is recommended to be the only service hosted by a particular Adaptive Processing Server in

anything beyond a small test environment.

OOM: Out of memory. When the Adaptive Processing Server or more precise the Java VM reaches the

upper memory limit the java dumps a heap file.

Java heap file: A heap dump is a file that contains a snapshot of the memory allocated by the Java Virtual

Machine. Contains technical information to find the root cause of a problem for developers and support

engineers.

Moreover this article will document that using the trace functionality on the java heap file it will show whatWeb Intelligence report or other reporting tool using BICS were the root cause of an out of memory incidentprocessed. Further to that how to interpret this information to improve reporting stability and performance.

When generating a Web Intelligence report based upon BICS which is in terms of BW a BEx Query the DSLBridge Service is communicating with BW. This implies that when executing or designing a Web Intelligence

report on a BEx Query there is transmitted BI Metadata from BW to Web Intelligence in order to allow usersto select attributes, key figures, hierarchies etc. With BICS connectivity, there isn’t any predefined universe.Web Intelligence needs then to generate a so called transient universe in runtime (“on the fly”). In theory thatwould imply that a BEx Query with a lot of metadata (key figures, restricted key figures, attributes,hierarchies etc.) will also need more time to build this transient universe. With the former BusinessObjectsUnivers design tool universes were pre-built and stored on the BusinessObjects Enterprise environment.

Figure 2: High level overview of Web Intelligence report run on BW using the BICS

The technical background after executing a Web intelligence report can be split into different parts:1. The BICS connection created in the CMC (or Information Design Tool) is used with Dimensional

Semantic Layer (DSL) Bridge.2. The DSL Bridge embeds with the DSL BICS layer.




6

3. DSL BICS layer uses the BICS driver (physical JAR-drivers stored on the operating system hosting

the SAP BO Enterprise server)

4. The BICS driver communicates with the Java Connector (JCO)

5. The JCO creates the RFC (Function Call) to SAP BW6. BW uses the BICS Backend functions to call the OLAP engine

7. The OLAP Engine fetches the data from the database.

Figure 3: Detailed technical description of APS & DSL Bridge Service




7

STEPS REQUIRED FOR EVALUATING THE JAVA HEAP FILE

The following steps describe how to collect data from the java heap file after an out of memory incident hasoccurred. If the java heap file isn’t generated automatically due to an incident where there was not an out of

memory the java heap file can also be generated manually.

Just before the Java Virtual Machine runs out of memory it tries to free up available memory – this is thework of the Garbage Collector (GC). In an undersized environment or where a user runs a maverick-querythe CPU usage is increased heavily when the GC works to prevent running out of memory (Xmx upper limitset in the CMC). If the GC isn’t able to free up memory the Java VM dumps the memory to a java heap file.

In short a java heap dump is a file that contains a snapshot of the memory allocated by the Java VirtualMachine. This file contains the very last activity just before the out of memory incident occurred and with a bitof detective work you can actually find the root cause and hopefully make the appropriate changes to avoidthis in future.

In the following it is described how to analyze the java heap file dump.

In order to find the biggest memory consumer in your APS process, you will first need to have a java heapdump.

Heap Dump generation:

If the process is reaching OOM (OutOfMemory) , it will generate a heap dump file automatically in“C:\Program Files (x86)\BusinessObjects\SAP BusinessObjects Enterprise XI 4.0\logging"

With the name java_pid[pid].hprof,e.g: java_pid11448.hprof

If the process is not reaching OOM, then you can generate manually a heap dump file :Choose in CMC->Servers and find the PID of the culprit server, e.g.: PID 10936Start a command line tool:cd “C:\Program Files (x86)\BusinessObjects\SAP BusinessObjects Enterprise XI

4.0\win64_x64\sapjvm\bin” jmap -dump:file=c:\dumpdemo.hprof 10936




8

Using Memory Analyzer Tool (MAT) to Analyze Java Heap Dumps:Once the heap dump file is generated (hprof extension), it can be analyzed by using the Memory AnalysisTool:

Get the tool from this link : http://www.eclipse.org/mat/

Figure 4: Overview of memory analyzer tool (MAT)

Evaluating the java heap log file using memory analyzer

The Eclipse Memory Analyzer is a fast and feature-rich Java heap analyzer that helps you find memory leaksand reduce memory consumption.

Use the Memory Analyzer to analyze productive heap dumps with hundreds of millions of objects, quicklycalculate the retained sizes of objects, see who is preventing the Garbage Collector from collecting objects,and run a report to automatically extract leak suspects.

Opening a java heap dump file using MAT:From the menu “File” chose Open Heap Dump…

Browse to the directory where you placed the java heap dump file and select the file.MAT will now give you a new menu to select from.

http://www.eclipse.org/mat/






9

Depending on the size of the java heap file dump it may take a few minutes to open.

You may also chose to run the report in background allowing to somewhat use the MAT.

Create a histogram to show the biggest memory consuming BICS queries:

As can be deducted by the screen shot below at the time of the out of memory issue this query was trying to

fetch 2.331.556 cells which would be a mighty large Web Intelligence report:




10

Instability or performance issue – key findings and technical background:Using the Java Heap dump provides detailed information on the Adaptive Processing Server, which is can be

containing multiple BO services including the DSL Bridge (BICS connectivity), is restarted possible due to outof memory (OOM) issues.

At the moment of the crash – the BW system had fetched approximately 2.3 million cells:




11

Most likely it crashed when it was trying to get the data back from BW via the BICS connector to Webintelligence. It is returning almost 1 GB of raw-data as can be seen from below.

Searching from the “BICSQueryViewWrapper” shows the actual query that was run against the BWdatabase:

Right click on the item as shown below and select “Show objects by class” ->”by outgoing references”

Which gives the following:




12

Drill down to “queryState” as shown:

On the left side a value appears with “..BICS_VIEW..” text. This value contains the exact statement that thiswhole exercise is all about.

To obtain the query statement right click on the value field and select “Copy” -> “Value” as shown on thescreen shot below:

Copy and paste the information extracted from the copied XML into your favorite text editor. In the following Iuse Notepad++




13

<Query> This shows which Query that was used to access the BW data.The safety belt “<BICS_DA_REULT_SET_MAX>” can be applied to reduce the risk of running out of memory

– please consult document: “SAP BusinessObjects increasing stability by setting limits on max. retrievablecells from SAP BW into Web Intelligence using BICS.” https://scn.sap.com/docs/DOC-31900 The Web Intelligence report that caused the problem is using the following BEx query:“SALES_ANALYSIS_BICS_QUERY_001”.

The extracted information also reveals that the following dimension attributes and measures were selected:

Variables (prompt that is performed on the BW system):

https://scn.sap.com/docs/DOC-31900







14

It seems the incident with restarting the Adaptive Processing Server happened with just one BW connectionopen – this together with the SQL Statement indicates that this report is an Ad Hoc Web Intelligence reportbuild upon the same BEx Query BICS connection as the one used when the APS restarted due to an out ofmemory issue. This very query was for training purposes also run on the BW side which also caused the BW

system to run out of memory!

In the next section it will be discussed how to avoid this kind of out of memory issues.

STEPS TO PREVENT OOM FROM HAPPENING AGAIN

“The usual suspects” based on the information gathered from the log file it can be determined that the out ofmemory in many case it was due to a “runaway query” generated by an Web Intelligence user whomselected millions of transactions to be returned. The way Web Intelligence works is that it will return all data

in a query to the micro cube stored on the SAP BO Enterprise server.

Optimizing the APS – see “How to APS”, http://scn.sap.com/docs/DOC-31711

Redesign the BEx Queries to avoid situations where users inadvertently can produce problematic queriesusing Web Intelligence. Ensure that end users will at some time make a Web Intelligence report based on allthe possible fields in the BEx Query BICS connection – that implies redesigning the BEx Queries used for AdHoc Web intelligence reports. A good strategy for BEx Query as data source for BO Web Intelligence shouldbe developed and in place before going to production. This will remove the possibility that the somewhatuntrained user can cause this problem again – trying to retrieve rows of data into a report. It is at the momentnot possible to stop a runaway Web Intelligence query based on BW from the SAP BO frontend. However itis highly advisable to limit the number of rows retrieved from BW using BICS connection – using the safetybelt feature.

Choose the right tool - product position: use a dedicated OLAP tool such as Analysis for OLAP or Analysis for MS Office. It is advisable to consider using other tools in the BO portfolio for ad hoc or dataexplorations. Crystal Reports could also be used for ad hoc analysis – crystal only retrieves pages of data

and is hence a better tool if the underlying amount of data in a report is considerable. Moreover SAP BOExplorer is also a choice to consider if the end user reporting requirements are of a more data exploringnature. The same goes for Analysis for OLAP which could also perform far better on a larger set of data in anad hoc reporting scenario using BICS. “Selecting the Right SAP BusinessObjects BI Client Product Based onyour Business Requirements for SAP BW Customers” https://scn.sap.com/docs/DOC-32449

Safety belt feature: Implement the safety belt to reduce that risk or having users creating run-away quiescausing out of memory issues. SAP BusinessObjects increasing stability by setting limits on max. retrievablecells from SAP BW into Web Intelligence using BICS. https://scn.sap.com/docs/DOC-31900

http://scn.sap.com/docs/DOC-31711















15

HOW TO INCREASE THE STABILITY OF SAP BO ENTERPRISE

In a situation where the evaluation of the Java heap log file revealed that the adaptive processing server wasrunning out of memory and the WebI query (or queries) that caused this were within what could be expected.

The reason for OOM could be due to lack of specific sizing or just the fact that SAP BO Enterprise wasinstalled with default setting without any post-configurations applied.

Right click on the AdaptiveProcessingServer and select Clone Server:




16

Clone the APS and name it to APS_DSL_Bridge:

Select the services for the new cloned server:

Select the DSL Bridge Service, Web Intelligence and Security Token Service for the new cloned server:




17

Assign the required memory to the APS_DSL_Bridge. In this example “-Xmx8g” (Gigabyte) memory isallocated to the java VM running the APS DSL Bridge Service.

Also remember to remove the DSL Bridge etc. from the original APS server – this can be done by stoppingthe APS server and choosing select services and remove the DSL Bridge Service, Web Intelligence andSecurity Token Service etc.




18

REFERENCE DOCUMENTATION

The following documents are referenced in this document:

DOCUMENT NAME LOCATION ISSUE DATE

Memory Analyzer softwarehttp://www.eclipse.org/mat/

SAP BusinessObjects increasingstability by setting limits on max.retrievable cells from SAP BW intoWeb Intelligence using BICS.

https://scn.sap.com/docs/DOC-31900 20-09-2012

“How to APS” http://scn.sap.com/docs/DOC-31711 Nov. 2012

SAP WIKI - APS http://wiki.sdn.sap.com/wiki/display/BOBJ/Business+Intelligence+4.x+APS+(Adaptive+Processing+Server)+Service+Dist

ribution

2012+

Sizing guidelines www.sap.com/bisizing. http://scn.sap.com/docs/DOC-33183

“Selecting the Right SAPBusinessObjects BI Client ProductBased on your BusinessRequirements for SAP BWCustomers”


SAP internal access only :

Wiki Memory Analyzer

https://wiki.wdf.sap.corp/wiki/display/METETG+-+Using+MAT+to+Analyze+SAP+ME+Heaumps

WebI. Troubleshooting BICS.https://wiki.wdf.sap.corp/wiki/display/BOBS12Y/WebIntelligence+troubleshooting#telligencetroubleshooting-HeapDumpgenerationonaspecificWebIntenceBICSworkflow







http://wiki.sdn.sap.com/wiki/display/BOBJ/Business+Intelligence+4.x+APS+(Adaptive+Processing+Server)+Service+Distribution





http://www.sap.com/bisizing






https://wiki.wdf.sap.corp/wiki/display/METG/METG+-+Using+MAT+to+Analyze+SAP+ME+Heap+Dumps





https://wiki.wdf.sap.corp/wiki/display/BOBJCAS12Y/WebIntelligence+troubleshooting#WebIntelligencetroubleshooting-HeapDumpgenerationonaspecificWebIntelligenceBICSworkflow



























© 2013 SAP AG. All rights reserved.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP

BusinessObjects Explorer, StreamWork, SAP HANA, and other SAP

products and services mentioned herein as well as their respective

logos are trademarks or registered trademarks of SAP AG in Germanyand other countries.

Business Objects and the Business Objects logo, BusinessObjects,

Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and

other Business Objects products and services mentioned herein as

well as their respective logos are trademarks or registered trademarks

of Business Objects Software Ltd. Business Objects is an SAP

company.

Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL

Anywhere, and other Sybase products and services mentioned herein

as well as their respective logos are trademarks or registered

trademarks of Sybase Inc. Sybase is an SAP company.

Crossgate, m@gic EDDY, B2B 360°, and B2B 360° Services are

registered trademarks of Crossgate AG in Germany and other

countries. Crossgate is an SAP company.

All other product and service names mentioned are the trademarks of

their respective companies. Data contained in this document serves

informational purposes only. National product specifications may vary.

These materials are subject to change without notice. These materialsare provided by SAP AG and its affiliated companies ("SAP Group")

for informational purposes only, without representation or warranty of

any kind, and SAP Group shall not be liable for errors or omissions

with respect to the materials. The only warranties for SAP Group

products and services are those that are set forth in the express

warranty statements accompanying such products and services, if

any. Nothing herein should be construed as constituting an additional

www sap com

How to Evaluate SAP BusinessObjects Out of Memory Generated Java Heap Files

Documents