FDRAnalyzer ® User Guide

FDRAnalyzer ® User Guide 3.5

1

FDRAnalyzer ® User Guide

Version 3.5

INTRODUCTION ............................................................................................................................................ 2 The FDRAnalyzer ........................................................................................................................................... 2 System requirements ...................................................................................................................................... 3

INSTALL AND USE THE FDRAnalyzer .......................................................................................................... 4 Step 1 of 6: Install the FDRAnalyzer ............................................................................................................... 4 Step 2 of 6: Add license key............................................................................................................................ 8 Step 3 of 6: Set medium log levels within application ..................................................................................... 8 Step 4 of 6 Change Microsoft Excel macro security settings .......................................................................... 9 Step 5 of 6: Configure the INI file .................................................................................................................... 9 Step 6 of 6: Invoke the FDRAnalyzer............................................................................................................ 12

UNINSTALL FDRAnalyzer ............................................................................................................................ 16 Steps to uninstall FDRAnalyzer .................................................................................................................... 16

TUNING AND TROUBLESHOOTING........................................................................................................... 17 Frequent instances of “NO FILE FOUND” .................................................................................................... 17 FDRAnalyzer runs slow when there are too many FDRs in BIN folder ......................................................... 17 How to re-generate a report for already analyzed crashes ........................................................................... 17 Does not generate Email Report................................................................................................................... 17

SUPPORT..................................................................................................................................................... 18


2

INTRODUCTION The guide walks thru the product features of the FDRAnalyzer and is meant for an audience that is conversant with the Siebel application. It primarily servers application support teams of Siebel application. The first part of the guide gives a brief introduction to the product and the systems requirement of the product. The second part of the guide gives a step-by-step approach to installing, configuring and using the product. The third part of the guide detail steps to uninstall the product. The last part of the guide helps in troubleshooting some commonly known issues.

The FDRAnalyzer Like any enterprise applications, Siebel application can be configured to suit company specific business processes that are part of the implementation. In order to achieve this, the vanilla Siebel code is modified and a compiled code is deployed on all Siebel servers. A typical Production environment where the final code is deployed may span multiple servers. Although every build goes thru a thorough quality assurance in its SDLC, it is nearly impossible to shake out all the bugs in a deployed application. This is also because of the fact that many bugs are a result of unique combination of application logic and incoming data. Inadequate error handling may cause these bugs to result in what we know as ‘crash’ in Production. The result of such a crash could be abrupt termination of the user sessions, unavailable functionality, unexpected error messages, loss of unsaved data. Siebel application runs as multithreaded process at the operating system level. Each of this process has a unique identifying id also called a PID (process id). Each PID maps to multiple threads which translates to tasks within the UI of Siebel application. A crash is a situation when a PID encounters an unhandled exception and terminates. A crashing PID writes a file called FDR capturing the context surrounding the exception. FDR is an abbreviation for flight-data-recorder. The FDR gives valuable information on what was happening inside the application at the time of the crash. The key to troubleshooting a crash is piecing together evidences surrounding a crash. The FDRAnalyzer is designed to address this need. Once configured, it does all of these:

- It parses the generated FDR file into a CSV format - It isolates the crashing thread - It sorts and filters events that lead to the crash and generates the context of the crash - It then gathers the supporting files like crash.txt and crashing component log file - It parses the component log for specific log-events into a separate file for ease of readability - It keeps all the evidences of the crash in a neat folder structure (sample screenshot of folder-

structure given below) - It summarizes the crashes in a HTML report - It emails the report to recipients (sample screenshot of email-report given below)


3

System requirements Operating System: Windows XP, Windows 2000, Windows 2003 SE (32-bit), Windows

2003 EE (32-bit) Office Application: Microsoft Office Excel 2003, Microsoft Office Excel 2007 Supported Siebel versions: Siebel Business Applications versions 7.7, 7.8 8.0,

Siebel Enterprise Applications versions 7.7, 7.8 8.0


4

INSTALL AND USE THE FDRAnalyzer

Step 1 of 6: Install the FDRAnalyzer Double-click setup.exe to invoke the installer. Click Next to proceed


5


6


7


8

Step 2 of 6: Add license key The FDRAnalyzer is licensed software and is governed by the End User License Agreement. The FDRAnalyzer User Guide gives instructions on how to add or change license keys. By default, when the application is installed, FDRAnalyzer runs in limited functionality mode with trial license. Kindly refer to steps in the “Configure the INI” section to change with permanent license keys. An extended trial license key may be obtained by writing to [email protected]. The extended trial license keys are provided on a case by case basis. A permanent set of keys may be purchased at http://fdranalyzer.weebly.com. The license-keys will be emailed to the address specified during purchase. For issues with regards to license-keys please write to [email protected] with transaction details. The FDRAnalyzer should be invoked only after configuring the INI file. Details on how to configure the INI file has been covered in the User Guide.

Step 3 of 6: Set medium log levels within application The FDRAnalyzer is designed to gather all artifacts related to a server-component crash in a single location. A successful crash analysis requires going through the artifacts and putting together a reproducible scenario. By default, an application is kept at low log-levels. It is done so because higher log-levels may result in increased stress on the IO subsystem, which in turn affects the performance of the application itself. However in order for the FDRAnalyzer to gather meaningful artifacts it is recommended to have the application running with medium log levels if not high log levels. This can be achieved by setting specific events to level 4 or 5. Here is an example script to set specific log-levels for an Object Manager component using SRVRMGR utility. # SET ALL LOGLEVELS TO ONE change evtloglvl %=1 for comp ComponentName # SET SPECIFIC LOG-EVENTS TO 4 change evtloglvl EventContext=4 for comp ComponentName


9

change evtloglvl MainThread=4 for comp ComponentName change evtloglvl TaskEvents=4 for comp ComponentName change evtloglvl ObjMgrBusCompLog=4 for comp ComponentName change evtloglvl ObjMgrBusServiceLog=4 for comp ComponentName change evtloglvl ObjMgrDataObjLog=4 for comp ComponentName change evtloglvl ObjMgrLog=4 for comp ComponentName change evtloglvl ObjMgrSessionInfo=4 for comp ComponentName

Step 4 of 6 Change Microsoft Excel macro security settings Settings in Office 2003 1. Open the Office 2003 or Office XP application in question. On the Tools menu, click Macro, and then

click Security to open the Macro Security dialog box. 2. On the Trusted Sources tab, click to select the Trust access to Visual Basic Project check box to turn on

access. 3. Click OK to apply the setting. You may need to restart the application for the code to run properly if you

automate from a Component Object Model (COM) add-in or template. Settings in Office 2007 1. Open the 2007 Microsoft Office system application in question. Click the Microsoft Office button, and

then click Application Options. 2. Click the Trust Center tab, and then click Trust Center Settings. 3. Click the Macro Settings tab, click to select the Trust access to the VBA project object model check box,

and then click OK. 4. Click OK.

Step 5 of 6: Configure the INI file Parameters in [Main] section InstallDir=C:\Program Files\FDRAnalyzer The default value is “C:\Program Files\FDRAnalyzer”. Please change the location, if it is installed at a different location. The specified directory needs to have read-write-permissions for the user executing the FDRAnalyzer executables. CrashAnalysisFolder=D:\sba80\SiebelAdminScripts\CrashAnalysis This parameter specifies the location where the analyzed crash files need to be organized. The FDRAnalyzer executable needs to have read-write privileges to this folder. By design, it creates sub-folders with a naming convention as “YYYYMMDD_SIEBSRVR_PID” format for each crash and gathers crash artifacts under it. CrashAnalysisROOTShare=\\FDRNLYZERHOST\CrashAnalysis This parameter is a share name that is used in the email reports sent out to the recipients. This directory may have read-only permissions for the user executing the FDRAnalyer executables. LicenseKey= This parameter is used to specify the temporary or permanent license keys. If an invalid license key is specified, the FDRAnalyzer executables aborts with error message in the logs. A value of “TRIAL” puts the application in limited functionality mode. In this mode, it analyzes only the first FDR of the server specified in Server1 parameter in the INI. FDRAnalyzerHostKey= This parameter identifies the machine name where the FDRAnalyzer is installed. This key can be obtained at purchase upon supplying the host name of the machine where the FDRAnalyzer will be installed.


10

Parameters in [FDRServers] section FDRServersLicensed=3 This parameter is used to specify the number of servers. FDRs on these servers are analyzed by the FDRAnalyzer. This parameter works hand-in-hand with the server-count parameter specified as “Server1”, “Server2” etc. For example: If “FDRServersLicensed=3”, then there should be three additional parameters namely Server1, Server2, Server3 as follows. Each in turn will give more details about the Siebel server covered in a later section. would mean adding three parameters as follows: Server1=SIEBHOST1;\\SIEBHOST1\D$\sba80\siebsrvr Server2=SIEBHOST2;\\SIEBHOST2\D$\sba80\siebsrvr Server3=SIEBHOST3;\\SIEBHOST3\D$\sba80\siebsrvr ServerN This parameter where N is substituted with a number is used to specify a server count.. This parameter specifies a) Siebel server name b) Semi-colon as a separator c) Read-write share to the siebsrvr folder of the Siebel server specified in (a). A read-write share is

required for the FDRAnalyzer executables. For example: Server1: OMSERVER1;\\HPDL58001\D$\sba80\siebsrvr Server2: OMSERVER2;\\HPDL58002\D$\sba80\siebsrvr Where OMSERVER1, OMSERVER2 are the Siebel server names, \\HPDL58001\D$\sba80\siebsrvr and \\HPDL58002\D$\sba80\siebsrvr are read-write shares. The executables accesses the..\siebsrvr\BIN, ..\siebsrvr\LOG and ..\siebsrvr\LOGARCHIVE directories as part of the analysis process. Parameters in the [SendEmail] section INCLUDECrashDateTime=TRUE An email report can be tailored to display only those fields that are required. The field INCLUDECrashDateTime displays the date and time of the crash. To make this field not appear in the email-report, set the value to FALSE. INCLUDEServerPID=TRUE An email report can be tailored to display only those fields that are required. The field INCLUDEServerPID displays the process id of the crashing process. To make this field not appear in the email-report, set the value to FALSE. INCLUDETaskThreadID=FALSE An email report can be tailored to display only those fields that are required. The field INCLUDETaskThreadID displays the thread id of the task. A thread-id is the OS equivalent of Siebel task-id. Though they are different, one task-id always maps to one-thread-id. To make this field not appear in the email-report, set the value to FALSE. INCLUDECrashedComponentLog=TRUE An email report can be tailored to display only those fields that are required. The field INCLUDECrashedComponentLog displays the name of the log file that caused the application to crash. To make this field not appear in the email-report, set the value to FALSE. INCLUDECrashDetailsAt=TRUE An email report can be tailored to display only those fields that are required. The field INCLUDECrashDetailsAt displays the name of the read-only shared location where the FDRAnalyzer has gathered the crash details. Typically this could be a value like


11

\\FDRNLYZERHOST\CrashAnalysis\20090428_OMSERVER1_7676 . To make this field not appear in the email-report, set the value to FALSE. SMTPServer=MAILSERVER201 This field is used to specify the SMTP server name. When configured, the FDRAnalyzer connects to this SMTPServer to send out the email-report SMTPPort=25 This field is used to specify the SMTP port number. When configured, the FDRAnalyzer connects to this SMTPport to send out the email-report [email protected] This field is used to specify the “From Address” of the email-report. EmailSubject=Analyzed PRODUCTION FDRs on $DATE This field is used to specify the “Subject-line” of the email-report. A $DATE makes the FDRAnalyzer pick up the date at run-time. This helps when the FDRAnalyzer is setup using scheduler. SendToEmailRecipients=2 This field is used to specify the number of recipients to the email-report. Depending on the value specified for this parameter, the EmailRecipientN parameter needs to be added to the INI in this section. For example: If SendToEmailRecipients=2, then EmailRecipient1 and EmailRecipient2 parameters need to be added to the INI and values specified. On the contrary to disable email-reports, set the value of SendToEmailRecipients=0. [email protected] The ‘N’ in this parameter needs to replaced with a number. This field is used to specify the email address of the recipients. Depending on the value specified for the parameter SendToEmailRecipients, the EmailRecipientN parameter needs to be repeated and values specified in this section of the INI. For example: If SendToEmailRecipients=2, then EmailRecipient1 and EmailRecipient2 parameters need to be added to the INI and values specified. Parameters in the [LogSearch] section SearchWAITIntervalINSeconds=5 This parameter is used to specify the wait time in seconds while searching for component log files. In most of the cases, the default 5 seconds has been found to be sufficient. However if the email-report generates too many entries of “NO FILE FOUND”, then this value needs to be increased. An increase in this value would mean longer execution time for the FDRAnalyzer to create the crash-analysis directories and compile the report. So this parameter requires tuning. MaxLogSizeINBytes=40000000 This parameter is used to specify the maximum file size (in bytes) of a log that should be copied. In many cases, a crash may be caused by a thread that has a log file size of 4 GB associated with it. In such cases, it may not make sense to copy the large log file to the crash-analysis folder. In such cases, FDRAnalyzer checks at this parameter to determine the threshold. When a decision to not copy is made, it creates a text file called LOG_FILE_TOO_LARGE.txt with details and location of the log file. In most of the cases, the default value of 40000000 (40 MB) for this parameter has been found to be sufficient.


12

Parameters in the [ParseLogs] section NoOfSiebelLOGEvents=3 This field is used to specify the number of Siebel log events that is to be parsed for each crash. Depending on the value specified for this parameter, multiple SiebelLOGEventN parameter need to be added to the INI in this section. For example: If NoOfSiebelLOGEvents =2, then SiebelLOGEvent1 and SiebelLOGEvent2 parameters need to be added to this section of the INI and values specified. On the contrary to disable log parsing, set the value of NoOfSiebelLOGEvents =0. SiebelLOGEvent1=EventContext The ‘N’ in this parameter needs to replaced with a number. This field is used to specify the log event for the Siebel component. Depending on the value specified for the parameter NoOfSiebelLOGEvents, the SiebelLOGEventN parameter needs to be repeated and values specified in this section of the INI. For example: If NoOfSiebelLOGEvents =2, then SiebelLOGEvent1 and SiebelLOGEvent2 parameters need to be added to this section of the INI and values specified.

Step 6 of 6: Invoke the FDRAnalyzer The FDRAnalyzer may be invoked two ways: By clicking the shortcut. Start > Programs > FDRAnalyzer > FDRAnalyzer Scheduling FDRAnalyzer reports using Windows scheduler The FDRAnalyzer can be configured to read the INI file and send reports at regular intervals using Windows scheduler. Follow these steps to do so.

1. Using the Windows Scheduler, setup a scheduled job. Start > Settings > Control Panel > Scheduled Tasks > Add a Scheduled task.

Follow the prompts to setup a scheduled job.


13


14


15

Review the advanced properties and press OK for the settings to take effect.


16

UNINSTALL FDRAnalyzer

Steps to uninstall FDRAnalyzer Follow these steps to uninstall the FDRAnalyzer 1. Navigate to the Add / Remove Programs. Scroll down to FDRAnalyzer and click “Remove”.

Click Yes when prompted with a confirmation.

The program gives a confirmation upon a successful un-install.


17

TUNING AND TROUBLESHOOTING

Frequent instances of “NO FILE FOUND” Issue: The FDRAnalyzer puts an entry of “NO FILE FOUND” in “CrashedComponentLog” field of the email-report in one of the following cases:

a. When it searches for the log file and does not find it. b. When the “SearchWAITIntervalINSeconds” value in the INI file is too small.

Possible resolution: The crashed component log search is faster when the machine on which the FDRAnalyzer is installed has a higher CPU speed. If the reason for the “NO FILE FOUND” is (b), then the “SearchWAITIntervalINSeconds” parameter may need some tuning. For slower searches, this parameter needs to be increased to a higher value. If the result is the same even after increasing the parameter value, then the cause of the error could be (a).

FDRAnalyzer runs slow when there are too many FDRs in BIN folder Issue: The FDRAnalyzer runs slow when there are too many FDRs in BIN folder. Possible resolution: The FDRAnalyzer is designed to go through all the FDRs in the ..\SIEBSRVR\BIN directory of the server. Hence it is highly advisable to have a separate cleanup job that deletes the old analyzed FDRs at regular intervals. An example cleanup job for deleting the FDRs in BIN directory using Windows FORFILES.exe is given below. net use X: /DELETE net use X: \\SIEBELHOST1\D$ D:\sba80\SiebelAdminScripts\LogCleanup\forfiles.exe -pX:\sba80\siebsrvr\bin -s -mT*.fdr -d7 -c"Cmd /C del /Q @FILE" D:\sba80\SiebelAdminScripts\LogCleanup\forfiles.exe -pX:\sba80\siebsrvr\bin -s -mcrash*.txt -d7 -c"Cmd /C del /Q @FILE"

How to re-generate a report for already analyzed crashes Issue: Many times it may be required to re-generate the FDRAnalyzer report for crashes that have already been analyzed. Possible resolution: When FDRAnalyzer analyzes a FDR it creates a folder “YYYYMMDD_SIEBSRVR_PID” and gathers the crash artifacts under it. It creates separate folders for each FDR file encountered. In order to re-analyze a FDR and re-generate the report, ensure the appropriate FDRs exist in the BIN folder and rename / delete the previously created YYYYMMDD_SIEBSRVR_PID analyzed folder. This will force the FDRAnalyzer to analyze the FDR, create a new YYYYMMDD_SIEBSRVR_PID folder and put in the report.

Does not generate Email Report Issue: The FDRAnalyzer analyzes the FDRs but does not generate the email report.


18

Possible resolution: Review the log generated by the FDRAnalyzer If there are no new FDRs generated since the previous time the previous run of the FDRAnalyzer, there will be no email-report generated. This will be reflected in the logs. Another reason for not receiving the email report could be incorrect SMTPServer and SMTPPort specified in the INI. Review the FDRAnalyzer log to look for error that may have been encountered while sending the emails out. Verify that you are able to telnet to the SMTPServer and SMTPPort without any problems.

SUPPORT The FDRAnalyzer UserGuide is a comprehensive step-by-step guide to installing, configuring, uninstalling and troubleshooting the FDRAnalyzer. It should be treated as the primary source of support. A 5 calendar day email support from the date of purchase of license keys is provided by BethSoft Corp. This support is completely complementary, is not part of the purchase-price and may be revoked at any time. Any additional support required may be purchased separately from BethSoft.

FDRAnalyzer ® User Guide

Documents