Distribution Monitor

Distribution Monitor

Cornelia Haase, Markus Eble

28.02.07

Distributing R3load processes among multiple application servers

SAP AG 2004, Title of Presentation / Speaker Name / 2

Motivation

Unicode conversion causes high CPU load especially for cluster tables

One application server is typically not sufficient to use full capacity of the source DB host

As a rule of thumb, you need about 1 GHz CPU speed per 1 MByte/sec source DB speed


Overview

DB serverNUC

DB serverUC

R3load-Export & ImportAppServer




Run R3load processes on multiple machines in parallel.


Distribution Monitor + Migration Monitor

Migration Monitor

Distributes tasks between multiple R3load processes on one machine

Synchronizes Export and Import

Controls package order

Time Analyzer

Collects start and end time of tasks from R3load logs

Distribution Monitor

Distributes R3load tasks between multiple machines

Controls package order

Constantly start time analyzer

Collects time analyzer results and log files from multiple machines


Communication Directory

Source DB

Target DB

AppServer1

Data Dir

AppServer2

Data Dir

commDir

Export

Export

Import

Import

The central communication directory contains control information for each machine (*.properties, *.STR).

For monitoring purpose log and status information is copied from each machine to the communication directory.

The transferred data are stored in local data dirs on each machine


Directory Structure

<communication dir>

R3ldctl R3szchk pkgSplit sapview <machine dir>

importInstallDir

exportInstallDir

netEx-changeDir

<local data dir>

<working directory>

data db

distribution_monitorcmd.properties

R3ta Info


Distribution Process

Preparation Phase

Sta

tus

Mo

nit

or

Export Phase

Import Phase

The preparation phase creates control information and Migration Monitor is performed only on one designated machine.

The export and import is started on each machine. The export and import can run in parallel.

Monitoring is possible at any time.


Phases

Central preparation (-p) Create and distribute STR files Create Migration Monitor command files Run as <sid>adm

Local execution of Export/Import (-e/-i) Fetch STR files to local data dir Start Migration Monitor locally Continuously copy status file to communication directory Run as <sid>adm

Central monitoring (-d) Preparation status per machine Package distribution Package status


Preparation Overview

SourceDB

*.STRDDL*.TPL *.EXT *.WHR IDX*.CMDIDX*.TSK

*.STR *.WHR

R3ldctl R3szchk R3ta

R3load

PackageSplitter Where

Splitter



SourceDB

*.STRDDL*.TPL *.EXT *.WHR IDX*.CMDIDX*.TSK

*.STR *.WHR

<commDir>/<machine>/*.STR;*.WHR

R3ldctl R3szchk R3ta

R3load

PackageSplitter Where

Splitter

DistributionMonitor



Create control files for R3load DDL<DBTYPE>.TPL – database command files

*.STR – R3load package with description of all contained tables

Cut R3load packages ( *.STR) into suitable pieces R3szchk (optional)

package splitter

table splitter

Distribute resulting packages ( *.STR, *.WHR) between machines


Export overview

SourceDB*.STRDDL*.TPL

*.CMD*.TSK

*.TOC*.00?

*.XML*.LOG

R3loadtask and cmdfile creation

R3load export

export_time.*TimeAnalyzer


Export overview

SourceDB*.STRDDL*.TPL

*.CMD*.TSK

*.TOC*.00?

*.XML*.LOG


R3load export

<commDir/machine>

export_time.*TimeAnalyzer

DistributionMonitor

DistributionMonitor


Export Overview

Copy R3load control files (DDL*.TPL, *.STR, *.WHR) to local data dump directory

Create R3load command and task files (*.CMD, *.TSK) for each package

Export each package (*.00?, *.TOC, *.XML)

Monitor state (*.TSK, export_state.properties)

Run time analyzer to collect package runtime(export_time.txt, export_time.html)

Copy monitoring information to communication directory


Import Overview

TargetDB*.STRDDL*.TPL

*.CMD*.TSK

*.TOC*.00?


R3load import


Import Overview

TargetDB*.STRDDL*.TPL

*.CMD*.TSK

*.TOC*.00?*.LOG


R3load import

<commDir/machine>

import_time.*TimeAnalyzer

DistributionMonitor


Import Overview

Create R3load command and task files (*.CMD, *.TSK) for each package

Import each package

Monitor state (*.TSK, export_state.properties)

Run time analyzer to collect package runtime(import_time.txt, import_time.html)

Copy monitoring information to communication directory


Status Monitor

Displays overall status

Displays preparation phase status per machine

Displays export/import progress and package status per machine

Displays package distribution


Distribution Monitor - Display


Distribution Monitor - Display


Trouble Shooting I

Check distribution_monitor_<mode>.log

From log file: Determine step that failed

Determine further log files to check for that step

Check error messages and warnings in these log files

If failing step is calling another tool and no log file for this tool is available: check availability of called tool

Check database connection (with –test option)


Trouble Shooting II

In case of Import/Export problems Determine which package and which table failed

Lookup if the affected package occurs in (and check if any warnings are written for this package)

distribution_monitor_prepare.log<comdir>/info/distribution.txt<comdir>/<machine> (file <package>*.STR*.WHR)migration_monitor_export.log<exportInstallDir> (file <package>*.LOG, *.TSK, *.CMD, *.XML)<comdir>/<machine>/netExchangeDir (file <package>*.SGN)<local data dir> (<package>*.00?, *.TOC, *.STR, *.WHR)migration_monitor_import.log <importInstallDir> (file <package>*.LOG, *.TSK, *.CMD, *.XML)


No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

Microsoft, Windows, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation. IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390, OS/400, iSeries, pSeries,

xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere, Netfinity, Tivoli, and Informix are trademarks or registered trademarks of IBM Corporation in the United States and/or other countries.

Oracle is a registered trademark of Oracle Corporation. UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group. Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks

of Citrix Systems, Inc. HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium,

Massachusetts Institute of Technology. Java is a registered trademark of Sun Microsystems, Inc. JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by

Netscape. MaxDB is a trademark of MySQL AB, Sweden. SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver and other SAP products and services mentioned herein as well

as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

These materials are subject to change without notice. These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.

Copyright 2005 SAP AG. All Rights Reserved

Distribution Monitor

Documents

presentation

collect package

copy monitoring

run time analyzer

cmd file creation

communication

sap group

distribution