12Type s to save the SOAP configuration and exit the configuration We will perform the SOAP configuration later when configuring the event synchronization
After you have configured the TEMS you must install the application support Use the following steps to seed the hub monitoring server Remember that seeding adds product-specific data to the monitoring server
1 Start the monitoring server by running the following command from the $CANDLEHOMEbin directory
In this command tems_name is the Hub TEMS name and pc is the product code for each agent whose data you want to send to the monitoring server (You can retrieve the information by executing the command cinfo -i)
Note If your Tivoli Enterprise Console is not yet set up you can skip the previous section by typing NO and perform the configuration later using the same procedure
When you are done with the Hub TEMS refer to Table 3-12 for installing the rest of the components
Table 3-12 How to install IBM Tivoli Monitoring 61 components in scenario 2
3217 Replacing a Hub TEMS server with a new oneFor any reason (hardware upgrade for example) if you want to replace your Hub TEMS server with a new one from the same platform or a different platform IBM Tivoli Monitoring 61 allows you to do so This section describes the procedure we followed to replace our scenario 1 implementation (Windows TEMS servers) with scenario 2 (UNIX TEMS servers)
1 Install configure and seed the Hub TEMS on the madrid and milan servers Depending on the platform you are using refer to ldquoInstalling and configuring a Hub TEMS on a Windows serverrdquo on page 69 or ldquoInstalling a Hub TEMS on a UNIX serverrdquo on page 169
2 Stop all Remote TEMS (copenhagen and edinburg) using either the Manage Tivoli Enterprise Monitoring Services console or for UNIX servers use the following command
itmcmd server stop tems_name
tems_name is the of the TEMS host by the server your are stopping its TEMS service
Components References
Install and configure the second UNIX Hub TEMS
ldquoInstalling and configuring the scenario 2 environmentrdquo on page 169
Install and configure the remote TEMS
ldquoInstalling a Remote TEMS on a Windows and UNIX serverrdquo on page 82
TEPS ldquoTivoli Enterprise Portal Server - TEPSrdquo on page 87
TEMAs ldquoTivoli Enterprise Monitoring Agentrdquo on page 95 and ldquoDeploying TEMA from command line interfacerdquo on page 124
TEP ldquoTivoli Enterprise Portal (TEP)rdquo on page 126
Warehouse Proxy agent ldquoWarehouse Proxy installation and configurationrdquo on page 130
Summarization and Pruning agent
ldquoSummarization and Pruning agent installation and configurationrdquo on page 139
Event synchronization ldquoEvent synchronization installationrdquo on page 147
Hot standby ldquoConfiguring the Hot Standbyrdquo on page 162
176 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
3 Stop the TEPS Warehouse Proxy agent and Summarization and Pruning agent from the Manage Tivoli Enterprise Monitoring Services console
4 Stop the Hub TEMS (cairo and helsinki) using Manage Tivoli Enterprise Monitoring Services console or itmcmd command for UNIX servers as describe previously
5 Configure the Remote TEMS (copenhagen and edinburg) to point to your new Hub TEMS (madrid and milan primary and Hot Standby TEMS respectively) Use the Reconfigure option on the Manage Tivoli Enterprise Monitoring Services console on UNIX servers or the following procedure on UNIX servers
itmcmd config -S -t tems_name
6 Configure the Warehouse Proxy agent and Summarization and Pruning agent to point to the primary and Hot Standby TEMS (madrid and milan) respectively This operation is performed using the Manage Tivoli Enterprise Monitoring Services console
7 Using the Manage Tivoli Enterprise Monitoring Services console configure the TEPS to point to madrid
Important Before pointing TEPS to the new TEMS you must save TEPS data using the migrate-exportbat facility which detects the TEPS database and creates the SQL file script that will be used to restore your TEPS services Use the following procedure to back up and import the data
Saving TEPS data
Use the following procedure to back up your TEPS database
ndash cd to CANDLEHOMECNPS
ndash Run migrate-exportbat
The migrate-exportbat facility stops the TEPS creates a saveexportsql and restarts the TEPS
Importing backup data
Use the following procedure to restore your TEPS initial state
ndash Import the data back into TEPS by copying the presentation files back to the CNP directory
ndash Copy the saveexportsql file to the CANDLEHOMEcnpssqllib directory and run migrate_importbat
If you are using Linux as your TEPS read $CANDLEHOME instead of CANDLEHOME
Chapter 3 IBM Tivoli Monitoring 61 installation and first customization 177
8 Configure cairo to become a Remote TEMS This is done using the Manage Tivoli Enterprise Monitoring Services console or itmcmd on a UNIX server This step can be skipped if you do not want to reconfigure any of your old Hub TEMS to a Remote TEMS
9 Configure agents on oslo and dakar to point to cairo and copenhagen the primary and secondary Remote TEMS respectively You do not have to reconfigure your agents to point to the new Hub this operation was done to check whether the remote TEMS cairo is working as expected
10Start the Remote TEMS (edinburg copenhagen and cairo) Use the Manage Tivoli Enterprise Monitoring Services console or the itmcmd command for UNIX servers
11From the Manage Tivoli Enterprise Monitoring Services console start the Warehouse Proxy agent Summarization and Pruning agent and TEPS
Notes
This procedure is recommended only if you want to change your platformrsquos Hub TEMS If you want to build a two-Hub TEMS environment you can install it from scratch
We were able to get all of the agents (from scenario 1) back up and running we had to restart a few agents to have them reconnected Others reconnected automatically
We lost our situations workspace and collections configuration because we did not save it to transfer it later on the new Hub TEMS
178 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
33 Uninstalling IBM Tivoli Monitoring 61In this section we uninstall IBM Tivoli Monitoring 61 both the whole environment and individual components
331 Uninstalling the entire IBM Tivoli Monitoring environment Use these procedures to remove the entire IBM Tivoli Monitoring environment
Uninstalling the environment on WindowsUse the following steps to uninstall IBM Tivoli Monitoring from a Windows computer
1 From the desktop click Start rarr Settings rarr Control Panel (for Windows 2000) or Start rarr Control Panel (for Windows 2003)
2 Click AddRemove Programs
3 Select IBM Tivoli Monitoring and click ChangeRemove
4 Select Remove and click Next
5 Click OK
6 After Tivoli Enterprise services have stopped you are asked if you want to remove the Tivoli Enterprise Portal database Click Yes
Tips
If you want to change your Hub TEMS environment back up the entire environment and plan the transition very carefully
If you are replacing one Hub TEMS with another one with the same platform and configuration (host name TCPIP configuration) you can back up the CANDLEHOME directory from the current Hub TEMS and restore it later onto the new Hub TEMS
If the new and old TEMS are installed on the same platform (such as both UNIX or Windows) you can back up your candle database from your current Hub TEMS and restore it later onto the new Hub TEMS You must copy the following files from your old HUB TEMS
Windows
CANDLEHOMECMSQA1db and QA1idx
UNIX
$CANDLEHOMEtablesTEMS_NAMEQA1db and QA1idx
Chapter 3 IBM Tivoli Monitoring 61 installation and first customization 179
7 Type the password for the DB2 administrator in the Admin Password field and click OK
A pop-up window indicating that GSKit is being uninstalled is displayed
8 Select Yes to restart your computer and click Finish
Uninstalling the environment on UNIXBefore executing the uninstallation procedure make sure that all IBM Tivoli Monitoring 61 components are shut down
1 Stop the agent by executing this command from $CANDLEHOMEbin
itmcmd agent stop pc
pc is the product code (lz ux ul um ui cj and so on)
2 Stop the TEMS by executing this command from $CANDLEHOMEbin
itmcmd server stop TEMS_NAME
3 Run the following command
uninstallsh
A numbered list of product codes architecture codes version and release numbers and product titles is displayed for all installed products
4 Type the number for the installed product that you want to uninstall Repeat this step for each additional installed product you want to uninstall (Example 3-24)
Example 3-24 Uninstalling the environment on UNIX
Mon Oct 10 150829 EDT 2005 User itmuser Group itmuserHost name edinburgitscaustinibmcom Installer Lvl 400 100CandleHome optIBMITM
Products available to uninstall
Num Product [ Code Platform VersionRelease Description ]1 cj li6243 v610r172 Tivoli Enterprise Portal Desktop Client2 lz li6263 v610r115 Monitoring Agent for Linux OS3 ms li6243 v610r215 Tivoli Enterprise Monitoring Server4 sh li6243 v610r215 Tivoli Enterprise Monitoring SOAP Server5 uf li6243 v610r100 Universal Agent Framework6 ui li6243 v610r194 Tivoli Enterprise Services User Interface7 um li6243 v610r229 Universal Agent
Enter number for a product to uninstall or EXIT to exit 1
180 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
Confirm cj li6243 v610r172 Tivoli Enterprise Portal Desktop Client OK to delete [yn] y
5 When finished restart the computer to complete the uninstallation
332 Uninstalling an individual agent or component
Use the following procedures to remove an agent or other individual IBM Tivoli Monitoring component from your computer
Uninstalling a component on Windows Use the following steps to remove a component on a Windows computer You can uninstall a single agent or the entire agent bundle (such as IBM Tivoli Monitoring for Databases)
1 From the desktop click Start rarr Settings rarr Control Panel (for Windows 2000) or Start rarr Control Panel (for Windows 2003)
2 Click AddRemove Programs
3 Do one of the following
ndash To uninstall a single IBM Tivoli Monitoring component such as the portal server or portal client (but not all components) select IBM Tivoli Monitoring
ndash To uninstall an agent bundle or a specific agent select the agent bundle
4 Click ChangeRemove
Notes
If for any reason the UNIX uninstallation is not successful run the following command to remove all IBM Tivoli Monitoring directories
rm -r $CANDLEHOME
If you are uninstalling the components using a user ID different from root user you might have errors like the following
rm cannot remove `etcrc0dK10ITMAgents1 Permission deniedrm cannot remove `etcrc1dK10ITMAgents1 Permission deniedrm cannot remove `etcrc2dS99ITMAgents1 Permission deniedrm cannot remove `etcrc3dS99ITMAgents1 Permission deniedrm cannot remove `etcrc4dS99ITMAgents1 Permission deniedrm cannot remove `etcrc5dS99ITMAgents1 Permission deniedrm cannot remove `etcrc6dK10ITMAgents1 Permission deniedrm cannot remove `etcinitdITMAgents1 Permission denied
Contact your system administrator to remove those files
Chapter 3 IBM Tivoli Monitoring 61 installation and first customization 181
5 Do one of the following
ndash To uninstall a specific agent or component select Modify
ndash To uninstall the entire agent bundle select Remove
6 Click Next
7 Do one of the following
ndash If you are uninstalling an agent bundle click OK to confirm the uninstallation
ndash If you are uninstalling an agent or component do the following
i For an agent expand Tivoli Enterprise Monitoring Agents and select the agent you want to uninstall
ii For a component select the component (such as Tivoli Enterprise Portal Desktop Client)
iii Click Next
iv Click Next on the confirmation screen
v Depending on the remaining components on your computer there might be a series of configuration panels Click Next on each
8 Click Finish to complete the uninstallation
9 Restart the computer to complete the uninstallation
Uninstalling a component on UNIX Use the following steps to remove a component from a UNIX computer You can uninstall a single agent or the entire agent bundle (such as IBM Tivoli Monitoring for Databases)
1 From a command prompt run the following command to change to the appropriate bin directory
cd $CANDLEHOMEbin
$CANDLEHOME is the path for the home directory for IBM Tivoli Monitoring
2 Run the following command
uninstallsh
A numbered list of product codes architecture codes version and release numbers and product titles is displayed for all installed products
Note When removing a specific component (ModifyRemove) do not unselect any component other than the one you are removing Deselecting any other component will uninstall it from the machine
182 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
3 Type the number for the agent or component that you want to uninstall Repeat this step for each additional installed product you want to uninstall
4 Restart the computer to complete the uninstallation
333 Uninstalling the TEC event synchronization
Use the following steps to uninstall the event synchronization from your event server
1 Stop event synchronization on the event server by running the following command
On Windows
lttec_installdirgtTMETECOM_TECbinstopcmd
On UNIX
lttec_installdirgtTMETECOM_TECbinstopsh
2 Run the following uninstallation program
On Windows
lttec_installdirgtTMETECOM_TEC_uninstuninstallerexe
On UNIX
tec_installdirTMETECOM_TEC_uninstuninstallerbin
tec_installdir is the location of the IBM Tivoli Enterprise Console installation
3 Follow the prompts in the uninstallation program
Notes
You can also run this uninstallation program in silent mode (by running the program from the command line with the -silent parameter) or in console mode (by using the -console parameter)
You must stop and restart the event server for these changes to take effect
If your event server is running on an HP-UX computer ensure that the _uninst and _jvm directories are successfully removed by the uninstallation program If they are not manually delete these directories
Chapter 3 IBM Tivoli Monitoring 61 installation and first customization 183
184 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
Chapter 4 Historical summarized data
This chapter describes the architecture planning and implementation of IBM Tivoli Monitoring 61 Historical Data Collection One of the primary features of the new IBM Tivoli Monitoring 61 product is the new Tivoli Data Warehouse (TDW) 21 In this chapter we discuss the overall architecture of how historical data is collected on IBM Tivoli Monitoring 61 agents how historical data is collected by a new warehouse server in Tivoli Data Warehouse V21 and how historical summarization and pruning occurs within the new Tivoli Data Warehouse V21 database We explain the details of the architecture of IBM Tivoli Monitoring 61 Historical Data Collection and Tivoli Data Warehouse V21 and show how Tivoli Data Warehouse V21 differs from the previous Tivoli Data Warehouse V1x architecture
Although the new Tivoli Data Warehouse V21 might be used in the future with other products in this chapter we focus only on the IBM Tivoli Monitoring 61 historical data
This chapter also uses real-world scenarios to plan design and configure IBM Tivoli Monitoring 61 and Tivoli Data Warehouse V21 historical collection The scenarios include the configuration of the historical data for the following agents Windows OS monitoring Linux OS monitoring AIX OS monitoring and DB2 on UNIX monitoring We also discuss some of the reporting tools that can be used with Tivoli Data Warehouse V21 database Finally we discuss how IBM Tivoli Monitoring 5x data is integrated into the new Tivoli Data Warehouse V21
4
copy Copyright IBM Corp 2005 All rights reserved 185
We discuss the following topics in this chapter
Overview
Architecture
Planning
Configuration
Reporting
186 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
41 OverviewThe new Tivoli Data Warehouse V21 has two new processes that collect summarize and prune data gathered from IBM Tivoli Monitoring 61 agents
A new Warehouse Proxy agent is the new data warehouse server The Warehouse Proxy agent collects data from the IBM Tivoli Monitoring 61 agents and stores the data in a relational database (DB2 Oracle or MSSQL)
The Tivoli Data Warehouse V21 data can optionally be configured to summarize and prune the historical data with another new process called the Summarization and Pruning agent
Figure 4-1 Historical data collection overview
These new processes are discussed in more detail in 42 ldquoArchitecturerdquo on page 191 In this overview section we focus on the primary differences between Tivoli Data Warehouse V21 and Tivoli Data Warehouse V1x The new Tivoli Data Warehouse V21 architecture is extremely different from the older Tivoli Data Warehouse V1x solution The primary differences between the two versions are discussed in the following three topics
Implementation differences Usability differences Scalability differences
Historical Data Historical Data CollectionCollection
Source Agent Warehouse Proxy Warehouse
24 Hours of Data
Summarization amp Pruning Agent
TEP
Chapter 4 Historical summarized data 187
411 Implementation differencesSummary of the implementation differences
Uses row-based schema versus star-based schema Detailed data is now stored in the Tivoli Data Warehouse V21 Tivoli Data Warehouse V21 supports Oracle MSSQL and DB2
One of the primary architectural differences between Tivoli Data Warehouse V21 and Tivoli Data Warehouse V1x is that Tivoli Data Warehouse V21 is based on a single database and a simple row-based schema Version 1x was based on a multi-tiered database structure with a central data warehouse database and a separate datamart database and it required a more complex process to load the databases using an ETL1 and ETL2 process
The Tivoli Data Warehouse V21 architecture has a much simpler data collection and summarization architecture The datamart database in Tivoli Data Warehouse V1x was based on a star schema A star schema is a series of fact and dimension tables that are arranged as a conceptual star At the core of a star schema is a technique called database normalization For more information see
httpenwikipediaorgwikiDatabase_normalization
Row-based schemas are much simpler and are basically stored as a ldquowhat you see is what you getrdquo arrangement Row-based schema are easier to manipulate than star schemas However star schemas are far more efficient for processing large amounts of data We cover this in 413 ldquoScalability differencesrdquo on page 190 and in more detail in 43 ldquoPlanningrdquo on page 209
Another difference between Tivoli Data Warehouse V21 and Tivoli Data Warehouse V1x is that Version 21 now stores detailed level monitoring data (RAW data) in the data warehouse The V1x data was always aggregated to the hourly level when it was stored in the data warehouse Version 21 also improves the way data aggregation occurs This version is more flexible in ways attributes can be aggregated and it always aggregates from the raw data (the detailed tables) This topic is discussed in more detail in 42 ldquoArchitecturerdquo on page 191 in this chapter
Finally the Tivoli Data Warehouse V21 database is not restricted to DB2 as the older version was Version 21 now support DB2 82 or higher Oracle 9 or higher and MS SQL Server 2000
188 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
412 Usability differencesSummary of the usability differences
Tivoli Data Warehouse V21 stores more than 24 hours of detailed data Detailed and summarized data reside in the same database New GUI (Tivoli Enterprise Portal) access to Tivoli Data Warehouse An improved time-to-value
The usability differences between Tivoli Data Warehouse V21 and Tivoli Data Warehouse V1x are significant One of the biggest new features with Version 21 is that customers now can access more than 24 hours of detailed data Users can configure a maximum time to keep detailed data that is architecturally unlimited (Planning considerations should apply) For example a customer can now keep 30 days of detailed data in Tivoli Data Warehouse V21 if they choose (see ldquoPlanningrdquo on page 209) Also in Version 21 the detailed data is stored in the same database as the summarized data so users no longer have to navigate to multiple interfaces to run reports as all of the data is in the same database and can be accessed through a common portal Access to all levels of data is vastly improved in Version 21 In Tivoli Data Warehouse V1x a user had to use the Web Health Console to get to the detailed data and use a reporting tool such as Alphabox Crystal Reports Brio or SAS to access the summarized data (the datamart) In Tivoli Data Warehouse V21 a customer can access all tables (detailed or summarized data) from the same repository using any RDBMS reporting tool or using the Tivoli Enterprise Portal
The IBM Tivoli Monitoring 61 new portal provides a very robust GUI The Tivoli Enterprise Portal (TEP) interface is used for querying and viewing all levels of the stored historical data IBM Tivoli Monitoring 61 provides seven agents out of the box with thousands of out-of-the-box queries The fact that the combination of IBM Tivoli Monitoring 61 and Tivoli Data Warehouse V21 gives the customer seamless access to all levels of the data is a major improvement over the Tivoli Data Warehouse V1x architecture
Finally IBM Tivoli Monitoring 61 and Tivoli Data Warehouse V21 provide an improved time-to-value proposition IBM Tivoli Monitoring 61 and Tivoli Data Warehouse V21 can be implemented in a matter of hours compared to a few days and more likely weeks with the old Tivoli Data Warehouse V1x Obviously more than a few hours should be dedicated to the planning of a Tivoli Data Warehouse V21 implementation and we discuss this in more detail in ldquoPlanningrdquo on page 209
Chapter 4 Historical summarized data 189
413 Scalability differencesSummary of the scalability differences
Data collection Planning Performance
The new Tivoli Data Warehouse V21 features of storing detailed data in the database and allowing more than 24 hours of data can provide a tremendous opportunity However these features can also create some new pitfalls if proper planning is not applied In this chapter we discuss some of the implementation and usability weaknesses that were encountered using IBM Tivoli Monitoring 5x with Tivoli Data Warehouse V1x
However the IBM Tivoli Monitoring 5x - Tivoli Data Warehouse V1x architecture handled data collection and retrieval very well In the Tivoli Data Warehouse V1x architecture data was collected and retrieved by IBM Tivoli Monitoring 5x from the endpoints using the Tivoli Framework MDist2 architecture When a user requested a Web Health Console screen that required detailed information a request was made to the gateway and then to the endpoint using MDist2 All data collections from the endpoints in IBM Tivoli Monitoring 5x are initiated from the endpointrsquos gateways and the data is uploaded into an RDBMS from a gateway This data collection process also makes use of the Frameworkrsquos MDist2 architecture
IBM Tivoli Monitoring 61 does not use the Tivoli Framework architecture and MDist2 to collect data In Version 61 if a user does a query against detailed data for less than 24 hours for multiple servers the overhead of the request could significantly affect the performance of the overall IBM Tivoli Monitoring 61 monitoring infrastructure In contrast IBM Tivoli Monitoring 5x and the MDist2 architecture always retreived and collected data using a multi-tier structure (for example endpoint- rarr gateway rarr RDBMS) MDist2 also has a very robust architecture for throttling and tuning data movement throughout an infrastructure
Probably a more important scalability difference is that customers can now collect an enormous amount of data in Tivoli Data Warehouse V21 The collection and reporting against this data can have an impact on the IBM Tivoli Monitoring 61 monitoring infrastructure In Tivoli Data Warehouse V1x all of the warehouse data was already aggregated to the hourly level on the endpoint and only the aggregated data was uploaded to the gateways The new Tivoli Data Warehouse V21 architecture uses one server (called the Warehouse Proxy agent) that collects detailed data directly from all endpoints This one server initiates requests from all of the agents to export data directly to the Warehouse Proxy agent using ODBC connections
190 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
User reporting can also negatively affect monitoring performance depending on the configuration settings In the Tivoli Data Warehouse V1x architecture the datamart was in a separate database from the operational data (the IBM Tivoli Monitoring RIM database) Now in Tivoli Data Warehouse V21 a single database is both the operational database and the data warehouse database In Version 21 multiple users can run the same report against hundreds of servers for millions of lines of data In Tivoli Data Warehouse V1x the data was summarized and also normalized using datamarts and star schemas Users might want to consider using an OEM tool to create OLAP cubes or datamarts in Tivoli Data Warehouse V21 to offload some of the reporting requirements
42 ArchitectureIn this section we discuss the architecture of Tivoli Data Warehouse V21 as it relates to IBM Tivoli Monitoring 61 monitored data The main topics that are discussed in this section are
Historical data architecture overview Historical data types Component flows Data tables and attributes Object definitions
421 Historical data architecture overviewThe IBM Tivoli Monitoring 61 Historical Data Collection architecture is comprised of three primary components The following components are used to collect data in the IBM Tivoli Monitoring 61 architecture
Warehouse Proxy agent Summarization and Pruning agent Tivoli Data Warehouse V21 (Tivoli Data Warehouse V21)
The Warehouse Proxy agentThe Warehouse Proxy agent is the bridge between the active monitoring system and the historical data repository It handles warehousing requests from all managed systems in the enterprise and uses ODBC to write the historical data to a supported relational database Only one Warehouse Proxy agent at a time can be configured and running in an IBM Tivoli Monitoring instance (Hub TEMS) The Warehouse Proxy can only connect successfully to a hub monitoring server
Summarization and Pruning agentThe Summarization and Pruning agent maintains the data within the data warehouse by aggregation and pruning data based on customer specifications
Chapter 4 Historical summarized data 191
The IBM Tivoli Monitoring administrator sets up how often to collect the detailed data what intervals to aggregate and prune on and how often to run the aggregation and pruning engine Typically the summarization and running process is scheduled to run once a day
Tivoli Data Warehouse V21 The Tivoli Data Warehouse database is an integral part of the solution It stores a large amount of attribute data and customers will want to host this data on existing database farms The database will be used by the TEP if historical data is to represented External reporting tools and other applications can access the data and operate off of this database
422 Historical data typesThere are two types of data stores for the IBM Tivoli Monitoring 61 Historical Data Component
Short-term data Long-term data
Short-term dataShort-term data is typically referred to in IBM Tivoli Monitoring 61 as data that is stored in binary files and is less than 24 hours old In the IBM Tivoli Monitoring 61 architecture short-term data can be configured to store the binary files locally on the Tivoli Enterprise Management Agent (TEMA) or it can be configured to store the binary files on the Tivoli Enterprise Monitoring Server (TEMS) This can be configured by a user by agent type In both cases (TEMA or TEMS) the binary data is considered short-term because it is only designed for 24-hour access When the Summarization and Pruning agent is configured it can be set up to prune this short-term data When the short-term data is successfully loaded into the Tivoli Data Warehouse V21 to the Warehouse Proxy agent it is pruned on TEMA or TEMS if it is older than 24 hours If the Warehouse Proxy agent is not configured to collect the short-term data then a user-defined pruning job must be implemented When we wrote this book and unless otherwise specified by the specific agent it was recommended that the location for the binary short-term data be located the TEMA This configuration option is discussed in 44 ldquoConfigurationrdquo on page 224 The binary short-term data will never be in aggregate or summarized format regardless of whether it is stored on the TEMA or the TEMS
Long-term dataLong-term data in IBM Tivoli Monitoring 61 is typically referred to as data that is older than 24 hours and has been collected up to the Tivoli Data Warehouse V21 RDBMS to the Warehouse Proxy agent The long-term data resides in
192 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
tables in the Tivoli Data Warehouse V21 database The long-term RDBMS tables contain detailed data and summarized data in the Tivoli Data Warehouse V21 The Summarization and Pruning agent can be configured to run every day to roll up data from the detailed level to hourly to weekly to monthly to quarterly to yearly The Summarization and Pruning agent also prunes the summarized tables (Figure 4-2)
Figure 4-2 Historical data types
423 Component flowsWhen Historical Data Collection is configured in IBM Tivoli Monitoring 61 a user can determine whether the short-term data (the binary 24-hour data) should be stored on the Tivoli Enterprise Monitoring Server (TEMS) or on the Tivoli Enterprise Monitoring Agent (TEMA) If the data is stored on the TEMA then each monitored machine will store binary files for all of the monitoring agents running on that system As we wrote this book collecting historical data on the TEMS for large-scale customers was not recommended ldquoPlanningrdquo on page 209 discusses this in more detail
In some cases it might be necessary to collect short-term historical data on the TEMS Some agents require this configuration it also might be necessary if there are firewall considerations If the short-term Historical Data Collection is
Historical Data Types
Real Time Data Short Term Historical Data
Long Term Historical Data
Agent
SQL Queries
Tivoli Enterprise Portal
Memory refresh every x minutes Binary File 24 hours life time
RDBMS
NT Attribute GroupsNT_Memory NT_Process NT_Disk NT_Processor
DB2 Attribute GroupshellipDB2_BF DB2_Lock DB2_Inst DB2_Partition
SQL Attribute GroupshellipSQL_DB SQL_Lock SQL_TBL SQL_
Candle Data Warehouse
Chapter 4 Historical summarized data 193
configured to collect on the TEMS the binary files for all monitored machines and their agents will be collected up to the TEMS This will create a single binary file for each type of monitoring attribute group for all machines and can become a single point of failure and cause reporting queries to run for a long time
When the Warehouse Proxy agent and Summarization and Pruning agent are installed and configured data is loaded from the TEMA or TEMS (depending on the location setting) to the Tivoli Data Warehouse V21 RDBMS When data is collected to the Warehouse Proxy agent tables are created in the Tivoli Data Warehouse V21 database When the historical collection is configured a user can specify how often to prune the raw data The default is seven days After the raw data has been loaded into the Tivoli Data Warehouse V21 tables data older than 24 hours will be pruned from the short-term binary files located on the TEMA or TEMS At any given time you could have 24 hours of short-term raw data on the TEMA or TEMS and detailed tables in the Tivoli Data Warehouse RDBMS that contains the same data When a request is made from the Tivoli Enterprise Portal (TEP) to perform a query that uses the timespan function data is retrieved from the binary file if the timespan is less than or equal to 24 hours A query performed from the TEP that uses a timespan greater than 24 hours will retrieve data from the Tivoli Data Warehouse V21 tables
Important The most recent 24 hoursrsquo worth of data comes from a binary file stored at the agent or at the Tivoli Enterprise Monitoring Server Beyond 24 hours the data is retrieved from the Tivoli Data Warehouse The TEMS determines where to get the data either from the agent if the data is less than 24 hours old or from the Tivoli Data Warehouse if the data is older than 24 hours If the query goes to an agent and retrieves a large amount of data it can consume a large amount of CPU and memory You can experience low system performance while a large amount of data is retrieved from the agents
194 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
Figure 4-3 shows the flow of historical data collected when the location is stored on the TEMA
Figure 4-3 Historical collection location TEMA
ITM 61 Component Model
TEPS
(Hub)TEMS
TEPDesktop
TEPBrowser
OS Agent OS Agent
Application Agent
Universal Agent
This is an example of a component model where the historical data location is
configured on the TEMA
Warehouse ProxyTDW 21
Summarizationamp Pruning
Agent
TEMABinary
File
TEMABinary
File
TEMABinary
File
Chapter 4 Historical summarized data 195
Figure 4-4 shows the flow of historical data collected when the location is stored on the TEMS
Figure 4-4 TM 61 Component model (historical collection location TEMS)
424 Data tables and attributesHistorical collection of data is based on attribute groups which are defined as groupings of attributes within a specific IBM Tivoli Monitoring 61 agent For example the IBM Tivoli Monitoring 61 Monitoring Agent for Windows OS has 42 attribute groups with more than 1000 attributes Each agent has a set of default attribute groups defined that can be configured easily for historical monitoring Additional attribute groups can be configured if needed There is a separate user guide for each supported IBM Tivoli Monitoring 61 agent that describes the agentrsquos attribute groups and attributes
ITM 61 Component Model
TEPS
(Hub)TEMS
TEPDesktop
TEPBrowser
OS Agent OS Agent
Application Agent
Universal Agent
This is an example of a component model where the historical data location is
configured on the TEMS
Warehouse ProxyTDW 21
Summarizationamp Pruning
AgentTEMABinary
File
Note As we wrote this book the shipped agent user guides were inconsistent regarding the documentation for describing the Tivoli Data Warehouse V21 schemas Currently the only accurate way to get the schema table names is by querying the databases or looking at the Object Definition Interchange (ODI) files ODI is described in 425 ldquoObject definitionsrdquo on page 207
196 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
Table 4-1 is an example of three IBM Tivoli Monitoring 61 Agents and their default attribute groups
Table 4-1 Default attribute group examples
Short-term binary tablesWhen Historical Data Collection is turned on in IBM Tivoli Monitoring 61 the default attribute groups can be configured to collect historical data (see 44 ldquoConfigurationrdquo on page 224) After the data collection starts the agent starts storing short-term binary tables on the TEMA or TEMS depending on the collection location that has been configured For example Table 4-2 on page 198
Agent Default attribute group
Monitoring Agent for Windows OS Network_InterfaceNT_ProcessorNT_Logical_DiskNT_MemoryNT_Physical_DiskNT_ServerNT_System
Monitoring Agent for UNIX DiskSystem
Monitoring Agent for Linux Linux_CPU Linux_CPU_AveragesLinux_CPU_ConfigLinux_DiskLinux_Disk_IO Linux_Disk_Usage_Trends Linux_IO_ExtLinux_Network Linux_NFS_StatisticsLinux_OS_Config Linux_Process Linux_RPC_StatisticsLinux_Sockets_StatusLinux_Swap_Rate Linux_System_Statistics Linux_User_Login Linux_VM_Stats
Monitoring Agent for DB2 KUDDBASEGROUP00KUDDBASEGROUP01KUDBUFFERPOOL00KUDINFO00KUDTABSPACE
Chapter 4 Historical summarized data 197
lists four agentsrsquo default binary file table names These are the names of the binary file tables as they appear on the TEMA or TEMS
Table 4-2 Short-term binary table names
Agent Binary table name
Monitoring Agent for Windows OS NETWRKINNTPROCSSRWTLOGCLDSKWTMEMORYWTPHYSDSKWTSYSTEM
Monitoring Agent for UNIX UNIXDISKUNIXOS
Monitoring Agent for Linux LNXCPULNXCPUAVG LNXCPUCON LNXDISK LNXDSKIOLNXDU LNXIOEXTLNXLOGINLNXNETLNXNFSLNXOSCONLNXPROC LNXRPCLNXSOCKSLNXSWPRTLNXSYSLNXVM
Monitoring Agent for DB2 KUD3437500KUD3437600KUD4177600KUD4238000KUDTABSPC
Note As we wrote this book all of the tables listed in Table 4-1 on page 197 and Table 4-2 on page 198 were the shipped defaults Some of the Linux default tables may change in the GA and post-GA releases of IBM Tivoli Monitoring 61 For example Linux_Process should be removed from the default list because it can collect an enormous amount of data
198 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
Each short-term binary file table also has an HDR file Every binary file table has an associated HDR file (for example NTPROCSSRhdr) The timestamp of the HDR file can be useful to determine the first time that data collection took place for that attribute group The timestamp on the tablename (that is the file without the hdr) indicates the last time data collection occurred for that attribute group Using the timestamps of these files can be helpful for troubleshooting purposes
The short-term binary tables are not accessed directly by a user The binary tables are only accessed from the TEP for queries of data less than 24 hours The binary tables are also in a proprietary format Although the tables cannot be acessed directly it may be helpful to know the names of the tables to determine whether short-term historical data is being collected and for troubleshooting The short-term tables can be found in the default IBM Tivoli Monitoring 61 install directory For example
For Windows
CIBMIBM Tivoli MonitoringtmaIBM Tivoli Monitoring6logs
For UNIX
optIBMIBM Tivoli Monitoringltplatform abbreviationgtlzhist
Examples of ltplatform abbreviationgt are li6263 for Linux and aix513 for UNIX
Long-term RDBMS tablesAt the core of the Tivoli Data Warehouse V21 is a single RDBMS database The Version 21ndashsupported databases are DB2 82 or higher Oracle 92 or higher and MS SQL Server 2000 When an attribute group is configured and has started historical collection of data a set of tables is created in the Tivoli Data Warehouse one detailed table and multiple summarization tables for each attribute group For example if yearly quarterly monthly weekly daily and hourly summarization are turned on for the NT_Memory attribute group the following tables will be created in the Tivoli Data Warehouse
ldquoNT_Memoryrdquo The detailed historical table for NT_MemoryldquoNT_Memory_Hrdquo The summarized hourly historical table for NT_MemoryldquoNT_Memory_Drdquo The summarized daily historical table for NT_MemoryldquoNT_Memory_Wrdquo The summarized weekly historical table for NT_MemoryldquoNT_Memory_Mrdquo The summarized monthly historical table for NT_MemoryldquoNT_Memory_Qrdquo The summarized quarterly historical table for NT_MemoryldquoNT_Memory_Yrdquo The summarized yearly historical table for NT_Memory
Note The platform abbreviation varies based on product and platform support (such as between 32-bit and 64-bit)
Chapter 4 Historical summarized data 199
Example 4-1 on page 208 shows how to use an SQL query to get a list of all table names Figure 4-5 is a list of the NT_Memory tables in the Tivoli Data Warehouse from the DB2 82 Control Center
Figure 4-5 Example of NT_Memory Detail and Summarization tables
Some attribute groups collect data for single-instance attributes and some attribute groups collect attributes for multiple-instance attributes The NT_Memory attribute group is an example of a single-instance attribute group The ldquoNT_Memoryrdquo detailed table has only one row per collection interval The
Note All Tivoli Data Warehouse V21 table names are created with quotation marks surrounding the table name When referencing historical data in the Tivoli Data Warehouse database you must use the double-quote character to ensure correct access to that data
200 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
Monitoring Agent for UNIX attribute group for disk monitoring creates a table called ldquoDiskrdquo The ldquoDiskrdquo attribute group collects data for UNIX file systems and is a good example of a multiple-instance attribute group The ldquoDiskrdquo detailed table will have multiple rows per collection interval with a row for each file system found on the specific agent
Figure 4-6 shows an example of the UNIX Disk attribute group with collected data in the Tivoli Data Warehouse Notice that the 1050929094542000 timestamp has eleven filesystems for that one collection (cycle)
Figure 4-6 UNIX ldquoDiskrdquo Table (multiple-instance) example
Example 4-1 on page 208 shows a detailed list of the Tivoli Data Warehouse V21 table names and instance types
Chapter 4 Historical summarized data 201
Detailed tablesAll of the detailed tables are based on a row-based schema Each attribute group that has Historical Data Collection turned on creates its own unique table and unique columns Attribute values in the detailed tables will store the actual raw values Figure 4-6 on page 201 shows a display of the UNIX Disk table and some of the detailed values that are stored All of the attribute groups and attributes are discussed in the IBM Tivoli Monitoring 61 specific agent monitoring guides However as we mentioned previously there is no consistent documentation describing the table schemas that are available with IBM Tivoli Monitoring 61 One of the ways to get more information about tables and columns is discussed in 425 ldquoObject definitionsrdquo on page 207
Most of the columns in the detailed tables are unique according to their specific attribute group However three common columns are important to know in order to understand the Tivoli Data Warehouse V21 architecture and are useful for generating reports These columns are
TMZDIFF
The time zone difference from Universal Time (GMT) This value is shown in seconds
WRITETIME
The time the record was written in the database The format of this timestamp is a 16-character value in the format cyymmddhhmmssttt where
ndash c = century
ndash yymmdd = year month day
ndash hhmmssttt = hours minutes seconds milliseconds
Note When an attribute group is configured and collection is started all of the definitions for that attribute group are common for all agents In IBM Tivoli Monitoring 61 you cannot filter historical collection by agents or groups of agents For example if the NT_Memory attribute group is configured to collect historical data then all Windows OS Agents will collect this attribute group You cannot exclude certain machines or groups of machines for historical collection Furthermore all summarization and pruning definitions will be in effect for all agents that the attribute group apply to In other words if ldquoNT_Memoryrdquo is configured to keep seven days of detailed data there will be seven days of detailed data for all Windows machines that have the Windows OS Agent deployed
202 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
Timestamp
This the date and time the agent collects information as set on the monitored system The format of this timestamp is the same 16-character value (cyymmddhhmmssttt) used for WRITETIME
The origin node field is another field that should be considered when working with the Tivoli Data Warehouse V21 architecture The origin node is typically the host name of the resource and is different depending on the agent type
Developers of agents should use certain general guidelines for the origin node field but some agents do not follow those guidelines In general the origin node is constructed as follows
The originnode may be of this form instancehostnametype
instance is optional
The delimiter usually is a colon
hostname is the machine name but it can also be a broker name (in case of MQ Series for instance)
type is the node type or product such as KNT for the Windows agent KUX for the UNIX agent and so on
Here are some examples
Monitoring Agent for Windows OS
The attribute for the monitored server name is Server_Name For example
PrimaryCAIRONT
Monitoring Agent for UNIX OS
The attribute for the monitored server name is Server_Name For example
istanbulitscaustinibmcomKUX
Monitoring Agent for Linux OS
The attribute for the monitored server name is Server_Name For example
istanbulitscaustinibmcomLZ
Monitoring Agent for DB2
The attribute for the monitored server name is Server_Name For example
DB2KLLAA9BUD
One way to get a list of all table names in the Tivoli Data warehouse is to query the TEPS database On the TEPS database machine or from an ODBC connection you can run the following SQL statements to get a list of all the installed detailed tables from DB2
Chapter 4 Historical summarized data 203
connect to teps use db2admin using passwordselect tabnamelongtableproduct from tepskfwhistdata
In these statements
tabname is the short-term binary file table name longtable is the detailed table name in the RDBMS product is the name of the associated agent
Summarized tablesIf summarization is configured for an attribute group then additional tables that include summarized data will be created in the Tivoli Data Warehouse Summarization is the process of aggregating the detailed data into time-based categories for example hourly daily weekly quarterly and yearly Summarizing data enables you to perform historical analysis of the data over time Along with summarization parameters pruning definitions can also be defined The Summarization and Pruning agent will create the summarized tables and perform the pruning process to remove old data The Summarization and Pruning agent can be configured to run a summarization and running process once a day When the Summarization and Pruning agent process (for example ksy610exe on Windows) is started it runs as a process on the system OS (Windows UNIX or Linux) This process will sleep and wakes up every 5 minutes to check whether the summarization and pruning run has been scheduled to kick off (The default schedule is once per day at 0200 am) If the summarization and pruning is scheduled to run within this 5 minute interval it will then start the summarization and pruning agent scheduled run against the Tivoli Data Warehouse V21 database The summarization portion of the run is a rollup process that aggregates data from the detailed tables to the specific summarization time-based tables (hourly daily weekly quarterly and yearly) The pruning portion of the run will remove data from the detailed and summary tables based on the configured pruning parameters The default pruning parameters are as follows
7 days of raw data 90 days of hourly data 12 months of daily data 2 years of weekly data 3 years of monthly data 3 years of yearly data
204 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
The names of the summarization tables will be the same as the detailed table name with an additional one-character identifier Depending on the summarization interval that is chosen for the particular attribute group the additional tables are created in the Tivoli Data Warehouse
Table 4-3 Linux CPU tables example
The attributes in the summarized tables are stored in a different format than the detailed table attributes When the attributed are aggregated in the summarized tables they are stored in different formats depending on the type of data they represent Eight aggregation behavior characterization types are used for aggregation the following five types are used most often
Behavior characterization types GAUGE
Attributes that are range-based numeric data These attributes are aggregated with MIN MAX AVG and SUM values from the detailed data to the appropriate summarization period There will be four attributes in the summarized table for each detailed attribute definition in the detailed table The original attribute name is prefixed with MIN_ MAX_ AVG_ and SUM_
Important You can run only one Summarization and Pruning agent even if you have multiple Tivoli Enterprise Monitoring Servers that are sharing a single Tivoli data warehouse database Running multiple Summarization and Pruning agents causes conflicts because the multiple instances would attempt to prune the data in the tables simultaneously The negative impact is that the configuration settings for the summarization and pruning periods has to be set only in one Tivoli Enterprise Monitoring Server that monitoring server controls how the data is summarized and pruned for all monitoring servers
Timespan Example
Detail Linux_CPU
Hourly Linux_CPU_H
Daily Linux_CPU_D
Weekly Linux_CPU_W
Monthly Linux_CPU_M
Quaterly Linux_CPU_Q
Yearly Linux_CPU_Y
Chapter 4 Historical summarized data 205
For example the ldquoLinux_CPU_Drdquo table would have the following attributes for the System_CPU attribute
ndash MIN_System_CPU
ndash MAX_System_CPU
ndash AVG_System_CPU
ndash SUM_System_CPU
COUNT
Attributes that have increasing numeric values with occasional resets (for example counts of x since hellip) These attributes are aggregated with TOTAL HIGH LOW and LATEST values from the detailed data to the appropriate summarization period There will be four attributes in the summarized table using the original attribute name prefixed with TOT_ HI_ LOW_ and LAT_ For example the ldquoLinux_System_Statistics_Hrdquo table would have the following attributes for the System_Uptime attribute
ndash TOT_System_Uptime
ndash HI_System_Uptime
ndash LOW_System_Uptime
ndash LAT_System_Uptime
Count type attributes use delta-based aggregation Delta-based aggregation algorithms calculate the delta between two intervals and use that number as the stored value For example if you have an attribute that is the total amount of cache hits since the system has been started then a delta-based calculation would compute the difference between each cycle interval At the end of the summarization period it would total all deltas store the high value store the low value and store the last value recorded IBM Tivoli Monitoring Administratorrsquos Guide SC32-9408 has more details about delta-based summarization
PROPERTY
Attributes that rarely change (for example total amount of memory or CPU speed) There will be one attribute in the summarized table using the original attribute name prefixed with just LAT_ For example the ldquoLinux_VM_Stats_Qrdquo (Memory) table would have the following attribute for the Total_Swap_Space attribute
LAT_Total_Swap_Space
PEAK
Attributes that are high-water marks or snapshot based There will be one attribute in the summarized table using the original attribute name prefixed
206 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
with just MAX_ For example the ldquoLinux_Swap_Rate_Yrdquo table would have the following attribute for the Peak_Swap_Space_Used attribute
MAX_Peak_Swap_Space_Used
LOW
Attributes that are low-water marks or snapshot based There will be one attribute in the summarized table using the original attribute name prefixed with just MIN_ For example the ldquoLinux_Swap_Rate_Yrdquo table would have the following attribute for the Low_Free_Memory attribute
MIN_Low_Free_Memory
The other three types that are rarely used are
SAMPLECOUNT
Attributes that are used to calculate the number of intervals that are sampled to get an average There will be one attribute in the summarized table using the original attribute name prefixed with just SUM
PDEL
Attributes that are deltas precalculated by the application (change over a period of time) These attributes are aggregated with a MIN MAX and SUM values
STATE
Not used at this time Generally an enumeration list of options referring to the condition of a resource (for example up down)
425 Object definitionsIn IBM Tivoli Monitoring 61 all attribute groups and attributes are defined as object definitions in files called Object Definition Interchange (ODI) files Most of the Tivoli Data Warehouse V21 schema information can be obtained by knowing how to interpret the ODI files The ODI files can be found on the TEPS server in the default IBM Tivoli Monitoring 61 install directory (for example cIBMIBM Tivoli MonitoringCNPS) The format of the ODI files are
docnnn
(nnn is the product identifier)
Note If a column name exceeds the RDBMS name length (for example DB2 is 30 characters) the Tivoli Data Warehouse creates an internal column name and stores the internal name and original attribute name in a table called WAREHOUSEID
Chapter 4 Historical summarized data 207
For example
docknt The ODI file for Windows OS agentdockux The ODI file for UNIX OS agentdocklz The ODI file for Linux OS agentdockud The ODI file for DB2 agent
Example 4-1 shows selected parameters from a Windows docknt ODI file
Example 4-1 docknt ODI file example
TABLE WTSYSTEMOBJECT NT_SystemOCCURS SingleOPGRP COM CONFNLSID KNT0000INDEX IRAKEY USERNAME OSTYPE VERSIONINDEX IRAKEY NETADDRESS NUMOFPRCSR PRCSSRTYPE PAGESIZE PCTTLPRIVTINDEX IRAKEY PCTTLPCSRT PCTTLUSERT CTXSWITCH FLECTLBYT FLECTLOPINDEX IRAKEY FLEDATOP FLEREADBTS FLEREADOP FLEWRTEBTS FLEWRTEOPINDEX IRAKEY PRCQUELNG SYSCALLSEC SYSUPTIME TTLINTSEC ALIFIXRATEINDEX IRAKEY EXCDISRATE FLOATERATE FILE VSAMWTSYSTEMREM System ObjectREM Index 002The System object type includes those counters that apply to allprocessors on the computer collectively These counters represent theactivity of all processors on the computer
ATTR Processor_TypeCAPTIONProcessorTypeCOLUMN PRCSSRTYPETYPE I4BEHAV PROPERTYNLSID KNT0020The type of the processors on the pc
ATTR _Total_User_TimeCAPTION_TotalUser_TimeCOLUMN PCTTLUSERTTYPE I4BEHAV GAUGERANGE 0-100ENUM Unknown=-1NLSID KNT0028The Total User Time is the average percentage
ATTR System_Up_TimeCAPTION System Up Time (Seconds)
208 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
COLUMN SYSUPTIMEPRINTF uTYPE I4BEHAV COUNTNLSID KNT0050Total Time (in seconds) that the computer has been operational sinceit was last started
The ODI files contain a lot of information describing IBM Tivoli Monitoring 61 objects In this chapter we describe only ODI files for the purposes of finding out information about the Tivoli Data Warehouse V21 schema Therefore we focus on only five ODI keywords
TABLE
Every attribute group is uniquely identified by a TABLE keyword in an ODI file In this example WTSYSTEM identifies the short-term binary file table name for the Windows OS agent After each TABLE keyword are multiple ATTR keywords (one for each attribute defined in that table)
OBJECT
The OBJECT keyword identifies the long-term RDBMS table name If the table name exceeds the RDBM name limit an internal name will be used in the RDBMS and the real name will be in the WAREHOUSEID table
ATTR
The ATTR keyword identifies the column names used in the RDBMS tables If the column name exceeds the RDBM name limit an internal name will be used in the RDBMS and the real name will be in the WAREHOUSEID table
TYPE
The TYPE keyword can be used to determine how the column will be stored in the RDBMS It does not describe the actual DDL but it can give an idea of whether the column is for example a string or integer
BEHAV
The BEHAV keyword describes the aggregation alogorithm used for summarization and pruning See ldquoBehavior characterization typesrdquo on page 205
43 PlanningIn this section we discuss the physical and logical planning considerations for implementing the Tivoli Data Warehouse V21 Before reading this section you
Chapter 4 Historical summarized data 209
should review the Chapter 2 ldquoArchitecture and planningrdquo on page 15 The following topics are discussed in this section
Physical configuration considerations
ldquoLogical configuration considerationsrdquo on page 214
ldquoConsiderations when implementing the Historical Data Gathering componentrdquo on page 224
431 Physical configuration considerationsWhen planning the physical layout for the new Data Warehouse components shipped with IBM Tivoli Monitoring 61 (Tivoli Data Warehouse V21) you have to consider the size of the enterprise Chapter 2 ldquoArchitecture and planningrdquo on page 15 discusses three main categories of enterprises
Small to medium enterprises (up to 400 agents) Large enterprises (up to 4000 agents) Huge enterprises (greater than 4000 agents)
When planning a Tivoli Data Warehouse V21 architecture the three main components to consider when implementing the physical layout of the enterprise are
Warehouse Proxy agent Summarization and Pruning agent Tivoli Data Warehouse Database Server
Small to medium enterpriseIn a small to medium enterprise it is acceptable to run a minimal Tivoli Data Warehouse configuration A minimal configuration could have the Warehouse Proxy agent Summarization and Pruning agent and the Tivoli Data Warehouse database server on the same machine The primary caveat to this is that in IBM Tivoli Monitoring 61 the Warehouse Proxy agent has to be on a Windows box Therefore if an enterprise wanted to run the Warehouse Proxy agent and database server on the same machine it would have to run on a Windows OS An alternative configuration could be to install the Tivoli Data Warehouse database server (the RDBMS server) on another platform (such as Linux or UNIX) If the database server is installed on a machine other than the Warehouse Proxy agent machine it is also a good idea to install the Summarization and Pruning agent on that same database server machine
Note Remember when planning the Tivoli Data Warehouse architecture that you can run only one Warehouse Proxy agent per Hub TEMS configuration and only one Summarization and Pruning agent for the entire enterprise (multi-Hub TEMS configurations)
210 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
Figure 4-7 is an example of a minimal configuration where the Warehouse Proxy agent Summarization and Pruning agent and the Tivoli Data Warehouse database server are all on the same machine
Figure 4-7 Small to medium topology
Large enterpriseIn a large enterprise the Warehouse Proxy agent should be separated from the Summarization and Pruning agent and Tivoli Data Warehouse database server In a large enterprise topology it is best to run the Tivoli Data Warehouse database server on its own dedicated machine It is also recommended that you run the Summarization and Pruning agent on the same machine as the Tivoli Data Warehouse database server
Historical Data Proxy Flow
T iv o li E n te rp r is e M o n ito r in g
A g e n ts (T E M A )
TDW DB
Warehouse Proxy (WPA)
Summarization amp Pruning Agent
TEMS DB
REMOTETivoli Enterprise
Monitoring Server (TEMS)
HUBTivoli Enterprise
Monitoring Server (TEMS)
TEMS DB
Tivoli Enterprise Portal Server
(TEPS)
TEPS DB
Tivoli Enterprise Portal (TEP)
Desktop ClientHTTP Browser
ODBC
Hot Stand-by HUBTivoli Enterprise
Monitoring Server (TEMS)
TEMS DB
Stand-by heartbeat
Chapter 4 Historical summarized data 211
Figure 4-8 is an example of a large enterprise configuration in which the Warehouse Proxy agent is on one machine and the Summarization and Pruning agent and the Tivoli Data Warehouse database server are on another machine
Figure 4-8 Large enterprise topology
Huge enterpriseIn a huge enterprise you must run multiple Hub TEMS due to the limitations of agents supported within a Hub TEMS Therefore in a multiple-Hub TEMS topology you can run multiple Warehouse Proxy agent servers Each Hub TEMS will have its own Warehouse Proxy agent All of the recommendations described in the large enterprise topology apply for a huge enterprise within each Hub TEMS The primary consideration in the huge enterprise is that you can only have one Summarization and Pruning agent in the entire enterprise You should not install the Summarization and Pruning agent in each Hub TEMS Only one Hub TEMS should have a Summarization and Pruning agent installed Therefore
HUBTivoli Enterprise
Monitoring Server (TEMS)
TEMS DB
Historical Data Proxy Flow
Tivoli Enterprise Portal(TEP)Desktop Client
HTTP Browser TEPS DB
Tivoli Enterprise Portal Server
(TEPS)
Tivoli Enterprise Monitoring
Agents (TEMA)
Tivoli Enterprise Monitoring
Agents (TEMA)
Tivoli Enterprise Monitoring
Agents (TEMA)
Tivoli Enterprise Monitoring
Agents (TEMA)
Event Synchronization amp
Forwarding
TEMS DB
REMOTETivoli Enterprise
Monitoring Server (TEMS)
TEMS DB
REMOTETivoli Enterprise
Monitoring Server (TEMS)
TDW DB
Tivoli Data Warehouse DB
(TDW)
Summarization amp Pruning Agent
Warehouse Proxy(WPA)
ODBC
Tivoli Enterprise Console (TMRTEC)
212 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments
the Summarization and Pruning agent must be installed in one specific Hub TEMS instance This Hub TEMS will act like a logical master There is no special configuration option for a master Hub TEMSmdashit is purely a logical consideration in regard to running the Summarization and Pruning agent
Figure 4-9 is an example of a large enterprise configuration where the Warehouse Proxy agent is on one machine and the Summarization and Pruning agent and the Tivoli Data Warehouse database server are on another machine Note the dotted circle around the Summarization and Pruning agent This represents the one logical master Summarization and Pruning agent associated with the one Hub TEMS
Figure 4-9 Huge enterprise topology
TDW DB
Tivoli Data Warehouse DB
(TDW)
Summarization amp Pruning Agent
Tivoli Enterprise Portal(TEP)Desktop Client
HTTP Browser
Tivoli Monitoring
Environment 2
Warehouse Proxy(WPA)
Tivoli Monitoring
Environment 1
Warehouse Proxy(WPA)
Tivoli Enterprise Portal(TEP)Desktop Client
HTTP Browser
Tivoli Enterprise Portal(TEP)Desktop Client
HTTP Browser
Historical Data Proxy Flow
Instance 1 Instance 2
Note SampPA needs to be logically associated with one master Tivoli Monitoring installation
Tivoli Enterprise Console (TMRTEC)
Chapter 4 Historical summarized data 213
Hardware considerations for the physical layoutThe following are hardware considerations for the physical layout
Warehouse Proxy agent server
ndash Processors Minimum of four for a large or huge enterprise
Database server
ndash Processors Minimum of four processors
ndash Disk drives 16 to 24 disk drives for a large or huge enterprise
Summarization and Pruning agent server
ndash Processors Minimum of four processors
ndash Memory Minimum 2 GB
Database server and Summarization and Pruning agent on the same machine
ndash Processors Minimum of four processors
ndash Memory Minimum 4 GB
ndash Disk drives 16 to 24 disk drives for a large or huge enterprise
432 Logical configuration considerationsThe primary considerations when planning the logical configuration and implementation of the Tivoli Data Warehouse V21 are the estimation of the database size and planning for the kind of data you want to keep in the Tivoli Data Warehouse V21 and how long you want to keep it (pruning)
Database sizingThe Tivoli Data Warehouse database has two primary types of tables detailed and summarization The detailed tables keep raw data The summarized tables keep aggregate data records (such as min max and total) The size of the Tivoli Data Warehouse can be estimated based on the total size of all of the detailed tables and all of the summarization tables When considering table sizes you also need to know how the data is stored in the tables In IBM Tivoli Monitoring 61 all attribute groups relate to tables in the Tivoli Data Warehouse Some are multiple instance tables which have more than one instance collected per interval (for example process) Multiple instance attribute tables have to be taken into consideration when estimating the size of the database
Figure 4-4 on page 215 shows the default pruning values for the detailed and summarized tables
214 Getting Started with IBM Tivoli Monitoring 61 on Distributed Environments