IBM ECM System Monitor - CENIT AG

IBM ECM System Monitor: Event forwarding to BMC Options and HowTo guide for event forwarding from IBM ECM SM to BMC Patrol

08.01.2014 CENIT AG Author: Alexander Pfau

Table of Contents

1 Intention .......................................................................................................................................... 2

2 Available options for IBM ECM SM event forwarding to BMC Patrol ............................................. 3

3 Setup of Event forwarding............................................................................................................... 4

3.1 Event forwarding via logfile ..................................................................................................... 4

3.2 Event forwarding via SNMP ..................................................................................................... 7

3.3 Event forwarding via MSEND .................................................................................................. 8

3.4 Event forwarding via EEIF Sink .............................................................................................. 23

4 Customizing and Filter options ...................................................................................................... 24

4.1 Event forwarding via logfile customizing .............................................................................. 24

4.2 Event forwarding via SNMP customizing .............................................................................. 26

4.3 Event forwarding via MSEND customizing ............................................................................ 27

4.4 Event forwarding via EEIF Sink .............................................................................................. 28

5 Recommendation .......................................................................................................................... 29

Januar 28, 2014 IBM ECM System Monitor: Event forwarding to BMC Page 2

1 Intention

This guide describes the different options for IBM ECM SM event forwarding to BMC Patrol

and how to setup and configure these options.

The event forwarding feature combines the monitoring and event processing features of IBM

ECM SM and the advantages of a Help Desk with 24/7 coverage. Here the IBM ECM SM can

be configured to monitor all IBM ECM systems, application components and logfiles and after

the ECM SM event processing is finished, all events will be post processed by the event

forwarding mechanism. That means, all events that are shown in the ECM SM web console

will be forwarded to an enterprise wide systems management tool that provides a single

console to an operations center.


2 Available options for IBM ECM SM event forwarding to BMC

Patrol

IBM ECM SM provides a large variety of event forwarding mechanism and options to other

Enterprise Systems Management tools, like IBM Tivoli, HP OpenView or BMC Patrol. For

IBM Tivoli and HP OpenView, ECM SM provides a native event integration, which means all

events will be forwarded using either Tivoli or HP native mechanism for sending the events to

the corresponding Tivoli or HP Event Server. For BMC Patrol other options have to be

considered.

In IBM ECM SM we have a few options for the event forwarding to BMC Patrol. Some of

these options are out of the box features and some options require customizing.

The out of the box options are:

Event forwarding via logfile

Event forwarding via SNMP

Event forwarding via EEIF

The option that requires customizing is:

Event forwarding via MSEND

The following pages contain setup information for the above mentioned event forwarding

options and how to customize and filter events. At the end a recommendation is also

included.


3 Setup of Event forwarding

Generally it’s possible to setup the ECM SM server initially with event forwarding. However,

almost all event forwarding mechanism can be enabled or disabled using the fixpack

installer. This guide describes how to use the fixpack installer, because it’s very similar to the

main installer and as soon as a fixpack is installed, the main installer cannot be used

anymore to change these settings – then the fixpack installer must be used.

3.1 Event forwarding via logfile

Event forwarding via logfile means, the ECM SM server creates a logfile that contains all

events for all monitored agents. The logfile will be created in a predefined directory at the

ECM SM server, the ECM SM agents will not create separate logfiles.

To enable the event forwarding via logfile, run the latest fixpack installer (here FP2 was

used) and click the “Next” button until the event forwarding configuration is shown.

As shown in the screenshot above, check the “Reconfigure Logfile forwarding sink settings”

checkbox and select “Enable Logfile Event Forwarding” in the first listbox.

In the next line, define the logfile name and location (e.g. a separate logging directory or a

logfile name that contains the date).

Then select “Default Single Line output format” in the listbox below and in the last listbox it’s

recommended to select “Report all except HARMLESS events” – if “WARNING” events

should be skipped as well, the last option should be selected.

With this configuration, the ECM SM server now creates a logfile in the defined directory that

contains one line per event now and “HARMLESS” events will not be created in the logfile.


This screenshot shows the created log

This screenshot shows how the logfile looks like

Here an example logfile entry:

Instance: cala_rex client processes_default;; Date: 2014-01-09T12:19:55Z UTC;; Hostname:

w2k8x64db297;; Message: # ERROR_installation: registry key

"SYSTEM\CurrentControlSet\Services\cala_rex_\DisplayName" not found;; Value: not_ok;;

Info: ;; Source: Cenit Advanced Logfile Adapter;; status: CRITICAL

All event fields (slots) are separated by “;;” and each event slot is described with a name,

like:

Instance (monitor instance)

Date (date and time of the event)


Hostname (hostname of the monitored server)

Message (monitor or logfile message)

Value (monitor value, can be numeric or alphanumeric)

Info (optional, additional information)

Source (event source)

Status (severity HARMLESS / WARNING / CRITICAL / FATAL are available

severities)

These slots can be used to fill the corresponding event slots in BMC.

Now a BMC agent must be installed and a BMC logfile adapter must be created and

configured for the BMC agent to read and forward the events to the BMC event server. The

BMC agent installation and configuration is not part of this guide.


3.2 Event forwarding via SNMP

Event forwarding via SNMP means, the ECM SM server collects all events for all agents and

send all processed events to a remote SNMP receiving tool.

To enable the SNMP event forwarding, run the Fixpack 2 installer and check the

“Reconfigure SNMP sink settings” checkbox. Select “Enable SNMP Event Forwarding” and

configure the SNMP server name (BMC server name), the supported SNMP version and the

SNMP communication port. The SNMP Enterprise OID is predefined.

After the Fixpack installer finished with exit code 0 and the ECM SM server

services/daemons are restarted, the ECM SM server starts sending SNMP traps to the BMC

server.

The screenshot shows SNMP Traps that are forwarded to BMC.


3.3 Event forwarding via MSEND

The event forwarding via MSEND uses a different approach. Here we have no option to

enable or disable the feature during the ECM SM server or fixpack setup. The MSEND event

forwarding option is a complex customizing and requires deep knowledge of ECM SM. The

example here is based on a Windows ECM SM server and all CRITICAL and FATAL events

will be forwarded as CRITICAL.

Prerquisites for the event forwarding via MSEND is:

MSEND is installed on the ECM SM server

The BMC cell configuration files are installed in the corresponding directory and

configured properly, so that MSEND can connect to the BMC server

After the BMC administrator has installed and configured MSEND correctly, we can start with

the ECM SM customizations.

change all monitors on all servers – in all message templates the whitespaces must

be replaced by “_” (due to technical limitation of the BMC MSEND tool that is used for

the BMC integration)

Open the Monitoring Manager and load the monitor configuration for each server to

redefine the message template in the monitors

Example before:

“ObjectStore StorageAreaStatus FNOS1”


After:

“ObjectStore_StorageAreaStatus_FNOS1”

After finishing the monitor adjustments, save the monitor configuration and restart the

CALA agent

After the CALA agent(s) is/are restarted, verify that the events with the new message

templates are shown correctly in the ECM SM event console and clean out all “old”

events

To verify that the BMC agent is installed correctly, check the BMC agent installation

directory (for this example it’s: d:\BMC (Agent) and d:\BMC\msend (MSEND))

Check that the MSEND binaries and configuration files are present as shown in the

screenshot below


The ECM SM server in this example is installed in c:\ECMSM, so please create a

C:\ECMSM\MSEND directory to keep all ECM SM related files in the same directory

Create a batch script bmc_event_forward.cmd in c:\ECMSM\MSEND with the

following content:

--------------------------------------------------------------------------

@ECHO OFF

REM

REM ECM SM to BMC Script

REM

REM PRODUCT: cala

REM

REM TARGET_DIR: D:/BMC/msend

REM

REM TARGET HOST: BMCSERV1


REM

REM These are the parameters from ECM SM

set MCELL_HOME=D:\BMC\msend\etc

set mc_tool=%1

set mc_host=%2

set mc_object=%3

set mc_object_class=%4

set mc_priority=PRIORITY_5

set severity=CRITICAL

set msg=%5

set ecmsm_error_id=%6

set mc_parameter_value=%7

set mc_ecmsm_source_type=%8

set mc_parameter=%9

REM static paramters for msend are:

REM -n = bmccell_BMCSERV1

REM -f = "D:\BMC\msend\etc\mcell.dir"

REM -j = "D:\BMC\msend\buf\ bmccell_BMCSERV1"

REM -a = ECMSM_Class

REM This will be executed:

D:\BMC\msend\bin\msend.exe -n bmccell_BMCSERV1 -a ECMSM_Class -r CRITICAL -b

"mc_tool=%mc_tool%;mc_host=%mc_host%;mc_object=%mc_object%;mc_object_class=%

mc_object_class%;ecmsm_error_id=%ecmsm_error_id%;mc_parameter_value=%mc_para

meter_value%;mc_ecmsm_source_type=%mc_ecmsm_source_type%;mc_parameter=%mc

_parameter%" -m %msg%

--------------------------------------------------------------------------

To adjust the script for another environment, open the script and edit the following:

o Replace the –n parameter value “bmccell_BMCSERV1” for the BMC server

target with the value that is defined in the mcell.dir file which is located in the

BMC msend\etc directory and save the file


Then adjust the ECM SM config file finca-cfg.xml in C:\ECMSM\server\cfg (open with

wordpad or any other proper editor)

Search for this part:

<cala-rex>

<property name="host" value="w2k8x64db297"/>

<property name="port" value="23802"/>



<property name="user" value="cala_rex"/>

<property name="password" value="000033193c003a1900"/>

</cala-rex>

Replace the user “cala_rex” with “admin”

Replace the password for the admin user (ECM SM logon admin user) with the new

encrypted password

To encrypt the password use the Monitoring Manager password encryption tool

Tools menu Password dialog

Enter the user name “admin” and the password for the admin user


the new part looks like this (please donot replace the server name in the “host”

property):

<cala-rex>

<property name="host" value="w2k8x64db297"/>

<property name="port" value="23802"/>



<property name="user" value="cala_rex"/>

<property name="password" value="0f071c0c1b1d5f5c5d00"/>

</cala-rex>

Save the file and restart the IBM ECM SM Server services (all)

Then go back to the ECM SM console and logon with the admin user and password

Go to the Rules and scripts administration console


In the Rules and Task scripts tab search for the streamproc_script and duplicate it

For the duplicate give the same name and the click on OK

2 script with the same name are shown now, one default and one custom


The custom script has priority now

Edit the custom script

Add the following content at the end of the streamproc_script (please check the

correct location as shown in the screenshot)

var sev = evt.get("SEVERITY");

if ((sev == "CRITICAL") || (sev == "FATAL"))

{

try

{

var scriptname = "bmc_forward_script";

evt = rulesEngineScriptPlugInSrv.executeScript(scriptname, null, evt);

if (evt === null)

{


logger.info("Event was discarded by bmc_forward_script.");

return null;

}

}

catch (e9)

{

printStackTrace("executeScript", e9);

}

}

Then click on “check syntax” and result should be this:

Then click on Apply


Then create a new script

To create a new script select the sample_script and rightclick “duplicate”

Give the name bmc_forward_script


Then open the new created script and replace the content with the following code

(please don’t forget to replace the ECM SM server name “w2k8x64…” that is used for

this example):

// +-------------------------------------------------------------+

// I I

// I (c) 1999-2011 CENIT AG (Stuttgart, Germany) I

// I All Rights Reserved I

// I Licensed Material - Property of CENIT AG I

// I I

// +-------------------------------------------------------------+

function processEvent(evt)

{

if ( evt.get("BMC_FORWARDED") == "true" )

{

logger.info("Event was discarded by bmc_forward_script.");

return null;

}

evt.put ("BMC_FORWARDED", "true");

logger.info("Rule script bmc_forward_script: Processing event msg=" + evt.get("MSG") + ",

sub_source:" + evt.get("SUB_SOURCE"));

// program execution

var mc_tool = "ECMSM";


var mc_host = evt.get("ADAPTER_HOSTNAME");

var mc_object = evt.get("APPLICATION");

var mc_object_class = evt.get("CLASS");

// var mc_priority = "PRIORITY_5";

// var severity = evt.get("SEVERITY");

var msg = evt.get("MSG").substring(0, 35);

var ecmsm_error_id = evt.get("ERROR_ID");

var mc_parameter_value = evt.get("VALUE");

var mc_ecmsm_source_type = evt.get("SOURCE_TYPE");

var mc_parameter = evt.get("STREAM");

logger.info("bmc_forward_script cmd startup before task: Start of cmd with the following

parameters:" + mc_tool + mc_host + mc_object + mc_object_class + msg + ecmsm_error_id

+ mc_parameter_value + mc_ecmsm_source_type + mc_parameter);

{

var retVal = taskExecSrv.execProgram(evt,

"w2k8x64db297.fsm45.com",

" c:/ECMSM/MSEND/bmc_event_forward.cmd",

[ mc_tool, mc_host, mc_object, mc_object_class, msg, ecmsm_error_id,

mc_parameter_value, mc_ecmsm_source_type, mc_parameter ]);

logger.info("bmc_forward_script cmd startup_1: Start of cmd with the following parameters:"

+ mc_tool + mc_host + mc_object + mc_object_class + msg + ecmsm_error_id +

mc_parameter_value + mc_ecmsm_source_type + mc_parameter);


evt.put("RET_VAL", retVal);

logger.info ("bmc_forward_script retVal:" +retVal);

}

return evt;

}

Check syntax and apply afterwards if no syntax problems are detected

Then click on Apply changes

Then restart the IBM ECM SM Server and CalaRex Server services and check if

events are arriving at the BMC console


It is recommended to create a scheduled Windows task that restarts the IBM ECM

SM CalaRex Service daily just to make sure, that the event forwarding works properly

Rolback:

o Remove the 2 created scripts from the ECM SM Rules and Script

Administration console


o After bmc_forward_script and the custom streamproc_script are deleted, click

on Apply changes and restart the ECM SM Server services.


3.4 Event forwarding via EEIF Sink

The event forwarding via EEIF Sink can also be enabled during the initial or during the

fixpack setup. Tivoli uses the same interface and here the ECM SM event integration works

properly, but at the moment this event integration mechanism is not tested for BMC and not

implemented somewhere although BMC officially supports the EEIF interface.

As soon as new information is available, this document will be updated with a “best

practices” setup information.

This screenshot shows the EEIF sink configuration in the fixpack installer


4 Customizing and Filter options

For all available sinks ECM SM offers customizing or filter options. The majority of the

customization or filtering can be done in the Server configuration console. Here all sinks

(logfile sink, EEIF sink and SNMP sink) have a separate mapping that can be used for

filtering and event information customization.

4.1 Event forwarding via logfile customizing

This screenshot shows the default file sink mapping. The default mapping discards

HARMLESS events (as configured during the fixpack setup). Here the original mapping can

be modified and other severities can be filtered as well.


This screenshot shows the file sink Target / Plugin properties. Here we can see which

template file is used for the logfile generating mechanism. The path is relative to the ECM

SM server install path (in our example it would be c:\ECMSM\server\cfg\.....tpl)

This template file can be adjusted, so that all available or required event slots can be used.

To adjust the template, open the template file in an editor and perform the changes:


4.2 Event forwarding via SNMP customizing

This screenshot shows the default SNMP Sink mapping. One of the usually changed

parameter is the community name in the Target / Plugin properties. In the mapping table

additional filter for the different severities could be applied, but never touch any of the cryptic

numeric codes or remove columns from the event mapping, this could cause invalid events.


4.3 Event forwarding via MSEND customizing

Due to the high customizing level of the entire MSEND forwarding feature, no additional

customizing options are explained here.


4.4 Event forwarding via EEIF Sink

This screenshot shows the default EEIF sink mapping. Due to a lack of experience related to

the BMC forwarding via EEIF sink, no customizing options or best practices

recommendations are mentioned here.


5 Recommendation

With the above explained event integration options to BMC, the IBM Enterprise Content

Management System Monitor is a very flexible tool and offers many options to integrate into

an existing Enterprise Systems Management architecture. Each explained forwarding

mechanism is successful implemented at customers all over the world (except the EEIF sink

based event forwarding)

The recommended event integration mechanism is either the logfile or the SNMP based

approach. Both options can be enabled or disabled with out of the box tools. Also the

customizing via the ECM SM web console is possible and almost all customer requirements

could be realized using the mapping features. Additionally to all technical reasons, the

implementation efforts for the SNMP and logfile based event forwarding are much lower.

The MSEND based event forwarding mechanisms benefit is the fact that the ECM SM server

uses a BMC tool to feed the events into BMC. However, here’s a lot room for mistakes during

the setup and each event causes a ECM SM task execution which costs additional

performance and could possibly make the task execution server too busy to execute tasks in

an emergency case. Also the ECM SM server event processing scripts might get updated

during fixpack installations or other ECM SM updates – here each time the modified

streamproc script needs a review to apply the latest changes. This causes additional update

efforts to the already higher implementation efforts.

IBM ECM System Monitor - CENIT AG

Documents