Top Banner
Europe Oracle|Siebel IBM Support Center Montpellier – PSSC Enterprise Integration Manager (EIM) Load Explanation with an example [Goal: populate a database with a huge number of rows to test an archiving solution integrated with Siebel] D ate : 13 th September 2006 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Siebel version: 7.5.3 Siebel Server OS: AIX 5.3 Database version: DB2 v8.2 1/68 [author] Christophe Menichetti
68
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Enterprise Integration Manager (EIM) Load Explanation with an example

[Goal: populate a database with a huge number of rows to test an archiving solution integrated with Siebel]

Date: 13th September 2006

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Siebel version: 7.5.3Siebel Server OS: AIX 5.3

Database version: DB2 v8.2Database Server OS: AIX 5.3

Web Server OS: AIX 5.3 (IBM HTTP Server)Siebel Tools version: 7.5.3.5

Web Client version: Internet Explorer 6.0.2800

1/59 [author] Christophe Menichetti

Page 2: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

2/59 [author] Christophe Menichetti

Page 3: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Table of Content _

INTRODUCTION------------------------------------------------------------------------------------------------------------3

PURPOSE OF THIS PAPER-----------------------------------------------------------------------------------------------3ORIGIN OF THIS PAPER--------------------------------------------------------------------------------------------------3SCENARIO OF THIS PAPER----------------------------------------------------------------------------------------------3

EIM OVERVIEW------------------------------------------------------------------------------------------------------------4

INTRODUCING ENTERPRISE INTEGRATION MANAGER----------------------------------------------------------4DATA MAPPING------------------------------------------------------------------------------------------------------------4INVOKING EIM-------------------------------------------------------------------------------------------------------------5

CONTEXT DESCRIPTION----------------------------------------------------------------------------------------------6

SIEBEL CONTEXT DESCRIPTION---------------------------------------------------------------------------------------6EXAMPLE CONTEXT DESCRIPTION------------------------------------------------------------------------------------7ARCHITECTURE CONTEXT DESCRIPTION----------------------------------------------------------------------------8

EIM EXPORT---------------------------------------------------------------------------------------------------------------10

EIM EXPORT OVERVIEW----------------------------------------------------------------------------------------------10EIM EXPORT CONCERN-----------------------------------------------------------------------------------------------10EIM EXPORT PROCESS------------------------------------------------------------------------------------------------15

First step: Preparing EIM Tables----------------------------------------------------------------------16Second step: Editing the configuration file-----------------------------------------------------17Third step: Running EIM export process----------------------------------------------------------19Fourth step: Checking results---------------------------------------------------------------------------22

EIM IMPORT---------------------------------------------------------------------------------------------------------------25

EIM IMPORT OVERVIEW----------------------------------------------------------------------------------------------25First step: Prerequisites------------------------------------------------------------------------------------25Second step: Editing the Configuration file-----------------------------------------------------30Third step: Running EIM import process----------------------------------------------------------32Fourth step: Checking results---------------------------------------------------------------------------34Fifth step: Solve problems and try again---------------------------------------------------------39

HOW TO MANAGE FOREIGN KEY MAPPING---------------------------------------------------------------------41What to do after?-----------------------------------------------------------------------------------------------41Example--------------------------------------------------------------------------------------------------------------41Action------------------------------------------------------------------------------------------------------------------42Managing foreign key mapping with Siebel Tools------------------------------------------44Resolving foreign key mappings-----------------------------------------------------------------------48IMPORTANT NOTE: organization columns--------------------------------------------------------53

3/59 [author] Christophe Menichetti

Page 4: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Introduction

Purpose of this paper

The purpose of this document is to give to reader an introduction to Siebel EIM Load. The main goal is to present a simple example of Siebel EIM load of data in order to start more complex EIM load of data in the future. With a simple example as document thread, the reader will be able to learn about such Siebel process and understand EIM Siebel mechanism.

Origin of this paper

We wanted to populate Siebel Database already existed with a lot of data to create a scenario in order to test an archive solution integrated with Siebel Application 7.5.3. Thus we need a huge scope of data and a huge numbers of rows to validate and test performance for this solution.

To populate our Siebel Database with random data, we were obliged to use Siebel EIM process. So we decided to take advantage of test performance exercise to execute another complex exercise: Siebel EIM load of user data.

Scenario of this paper

First we will describe EIM basis and we will continue with EIM data export description.

Then we will explain our first simple example of EIM import and we will describe all errors and difficulties encountered.

To finish we will describe the overall EIM exercise completed.

4/59 [author] Christophe Menichetti

Page 5: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

EIM Overview

Introducing Enterprise Integration Manager

In Siebel Application User data is populated and managed by you. User data is stored in one or more tables in the Siebel Database and it implies a lot of relationships between these tables. It is obviously complex to populate user data using SQL. First reason is referential integrity (which is maintained programmatically through ROW_ID) of Siebel data.

Siebel offers you processes to import, delete or export of data into tables. This process called EIM uses interface tables acting as a staging area between the Siebel Database and other data sources. Once data entered in EIM tables, EIM task performs data load into Siebel tables, according to Siebel System rules.

Data Mapping

Data Mapping determines which Siebel table columns will store external source data and which interface table columns will be used to import from source to destination. Data Mapping is Business Analyst job and is the more complex and important part in EIM process. Once you located correct EIM tables and its columns, you just have to run EIM task to import successfully you user data.

5/59 [author] Christophe Menichetti

Page 6: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Invoking EIM

All interface tables have 3 columns, which must be populated in addition to the data that was mapped. A number to identify all records that should be processed in the batch (IF_ROW_BATCH_NUM) and an unique number to identify the record in the batch (ROW_ID, which is NOT the same row id exposed previously in this document), the third one is a status. The EIM batch (or task) is using a configuration file (.ifb file) to specify the batch, order and type of processing.

6/59 [author] Christophe Menichetti

Page 7: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

7/59 [author] Christophe Menichetti

Page 8: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Context Description

Siebel context description

We use Siebel Finances Services Application (version 7.5.3) integrated with an archive solution called Optim Archive For Siebel. This solution allows users to archive old data in their Siebel Application to gain performance and space in production database for example. The aim of this document is not to describe this archive solution.

Thus, we can archive Siebel activities, opportunities and service requests.

We decided to work with Siebel Activities and to import a huge set of user data concerning activities.

8/59 [author] Christophe Menichetti

Page 9: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Example context description

As it is written before in the document, we decided to, first export Siebel Data to better understand EIM mechanism before importing Data into Siebel Tables. To do that, we chose an activity acting as sample for our export exercise and as template for our future import exercise.

9/59 [author] Christophe Menichetti

Page 10: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

During almost EIM exercises, which will be described, we worked with this record, these applets (one list applet and one form applet), this view and this screen.

With Siebel help menu of your browser client, you can obtain such information.

Hereafter you can see all Siebel information about our parameters set we worked with.

Screen: Activities Screen

 

View: Activity List View [Cache Mode : Not Cached]

 

Business Object: Action

 

Applets: Applet[0]: Activity List Applet With Navigation; Applet[1]: Activity Form Applet;

 

Business BusComp[0]: Action; BusComp[1]:  

10/59 [author] Christophe Menichetti

Page 11: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Components: Action;

Architecture context description

To be as complete as possible, we are going to give an architecture installation behind Siebel Application we are using during the whole document. We used a client windows machine (IBM Personal Computer) with Siebel Client (internet explorer), with remote connection software (putty software) to connect to Siebel Server and with DB2 remote connection tools (as Control Center or/and Command Center) to work with remote database.

For Siebel and database, we used a power5 server (p570) with AIX 5.3, partitioned into 3 partitions. One partition for Siebel Web Server, Siebel Server and gateway, one another for Optim Archive Solution and one last for Database Server (DB2 v8).

11/59 [author] Christophe Menichetti

Page 12: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

EIM Export

EIM Export Overview

To export data, EIM reads the data in the Siebel database tables and places the information in the appropriate EIM tables. You can then copy data from the EIM tables into another database. The export process generally populates the applicable EIM table with a row for every Siebel base table row encountered. As a consequence, where EIM tables have mappings to multiple Siebel base tables, one export operation can generate multiple rows within the EIM table governing the rows encountered within the Siebel base tables.

EIM Export Concern

At the start point, we know which row we want to export: record with description ‘ffr ffr’. But our concern is to know where (which EIM tables) our record will be exported. So we must find which EIM table will be used before exporting any Siebel Data.

We know also which view and applet are concerned (see below). So we can guess which Siebel Table is concerned using Siebel Tools:

Launch Siebel Tools shortcut

12/59 [author] Christophe Menichetti

Page 13: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

With following properties:

C:\sea752\tools\BIN\siebdev.exe /c "c:\sea752\tools\bin\enu\tools.cfg" /U sadmin /P sadmin /d ServerDataSrc /s C:\sia753.wri

13/59 [author] Christophe Menichetti

Page 14: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

14/59 [author] Christophe Menichetti

Page 15: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Thus we are able to know the base table of our record with Table field in Tools.

We are going to search EIM tables mapping with Siebel table ‘S_EVT_ACT’

15/59 [author] Christophe Menichetti

Page 16: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

We obtain the EIM Table list of EIM tables mapping with ‘S_EVT_ACT’ table

NOTE: the red lines represent old EIM tables with old name convention (name ending with IF extension) and they are inactive. Now EIM tables beginning with EIM extension.

Now we are going to focus on EIM_ACTIVITY table.

16/59 [author] Christophe Menichetti

Page 17: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

EIM Export Process

To export tables of data, EIM performs a sequence of tasks. Each task involves multiple passes; at least one pass is required for each EIM table included in the process.

To export data to EIM tables, EIM performs the following steps:

1 EIM initializes EIM tables for export

If CLEAR INTERFACE TABLE in the configuration file is TRUE, all rows with the specified batch number are deleted. Otherwise, a warning is issued if rows already exist with the specified batch number

2 It uses export parameter expressions in the configuration file to locate and export table rows:

■ If EXPORT ALL ROWS is TRUE, ignore any EXPORT MATCHES parameters and export all rows.■ If EXPORT ALL ROWS is FALSE; use EXPORT MATCHES parameters to locate specific rows.

Set IF_ROW_STAT to EXPORTED for rows that are successfully exported.

3 For parent tables, EIM locates child table rows and exports them to their corresponding EIM tables

Go to Configuration file section further, to learn more about these different parameters exposed above.

17/59 [author] Christophe Menichetti

Page 18: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

EIM Export Steps

An EIM export process is split into the following steps:

1. Preparing EIM Tables for Export Process2. Editing Configuration file for Export Process3. Running Export Process4. Checking result5. Extracting Data from EIM tables (here optional)

First step: Preparing EIM Tables

Before you initiate an export process, you should verify that rows do not contain an IF_ROW_BATCH_NUM matching the batch number you plan to use. If such rows do exist, you should either make sure that they do not contain data you need to preserve, or change the batch number for the export process. In each row that you are exporting, you may also want to set the IF_ROW_STAT column to FOR_EXPORT.

If EIM finds a row whose IF_ROW_BATCH_NUM matches the batch number for this export process, it does one of the following:

■ Clear the row if the CLEAR INTERFACE TABLES parameter is set to TRUE in the EIM configuration file

■ Issue a warning if the CLEAR INTERFACE TABLES parameter is set to FALSE in the EIM configuration file

Note also the following thing:

The values for the LAST_UPD and CREATED columns in the EIM tables always contain the values for the LAST_UPD and CREATED columns from the target base table. For example, if you use the EIM_CONTACT interface table to export data from the S_CONTACT and S_ADDR_PER base tables, the values of the EIM_CONTACT.LAST_UPD and EIM_CONTACT.CREATED columns contain the data from the S_CONTACT.LAST_UPD and S_CONTACT.CREATED columns, respectively.

18/59 [author] Christophe Menichetti

Page 19: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Second step: Editing the configuration file

EIM reads a configuration file that specifies the EIM process to perform (import, update, merge, delete, or export) using the appropriate parameters. The EIM configuration file (the default file is default.ifb) is an ASCII text file of extension type .IFB that resides in the Siebel Server/admin directory. Before you can run an EIM process, you must edit the contents of the EIM configuration file to define the processes for EIM to perform.

The EIM configuration file begins with a header section used to specify global parameters that apply to all process sections defined later in the file. Following the header section, there must be at least one process section with its associated parameters. Some process section parameters are generic for all EIM processes. Other process section parameters are specific to a particular EIM process, such as import.

In our case, the configuration file is the following:

[Siebel Interface Manager] // mandatoryUSER NAME = "SADMIN"PASSWORD = "SADMIN"PROCESS = Export EIM activities test #1 // mandatory

[Export EIM activities test #1] // the same name as process field value before

TYPE = EXPORT // mandatoryBATCH = 100 // mandatoryTABLE = EIM_ACTIVITY // mandatoryEXPORT ALL ROWS = true // to import row even in child table

All configuration file header must begin with “[Siebel Interface Manager]” and must contain “PROCESS = ‘xxxx’” field.

Then configuration file section “[xxxx]“ must contain a type (export, import…), a batch number “BATCH = 100 “ and the name of EIM table which is used (“TABLE = EIM_ACTIVITY “).

The specific parameters for export process are the following:

19/59 [author] Christophe Menichetti

Page 20: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

20/59 [author] Christophe Menichetti

Page 21: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Third step: Running EIM export process

On each pass, EIM processes one EIM table and performs a particular action on all rows in that table for that batch. Most passes affect only the EIM tables temporary columns (columns with name starting with ‘T_’); for example, resolving foreign keys. There are two methods for running an EIM process:

1. Using the graphical user interface (GUI)2. Using command line

The easiest way to run EIM process is to use GUI, but in our example, we are using command line and Siebel Server manager.

1. Put into SIEBELHOME/siebsrvr/admin the configuration file (named here testExportActivities.ifb)

2. Start the server manager in command line (command srvrmgr)

[For information on srvrmgr program, see Siebel System Administration Guide.]

srvrmgr /g localhost /e sia753_db2 /s part1_db2 /u sadmin /p sadmin

For your information, hereafter you can find the help guide

21/59 [author] Christophe Menichetti

Page 22: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

You can enter server manager command:

3. Run the following command to execute the EIM task

Run task for comp EIM with config= testExportActivities.ifb, errorFlags=1, SQLFlags=8

22/59 [author] Christophe Menichetti

Page 23: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

The enterprise component is EIM and the task is launched with following configuration:

testExportActivities.ifb => the configuration file errorFlags=1 => records details regarding failed rows SQLFlags=8 => records summary SQL

You can see all possible parameters for component EIM with the following command:

List parameters for comp EIM

4. A message with completed status will be displayed

23/59 [author] Christophe Menichetti

Page 24: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Fourth step: Checking results

When an export process ends, you should carefully check the results to verify that data was successfully exported. During each export process, EIM writes comprehensive status and diagnostic information to several destinations.

First, you can verify EIM logs in enterprise folder log:

/sia753_db2/siebsrvr/enterprises/sia753_db2/part1_db2/log

You can edit EIM log:

Then you can use your SQL tools (here DB2 Command Center) to check results.

24/59 [author] Christophe Menichetti

Page 25: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Launch DB2 Command Center

Connect to Siebel 7.5.3 Database

Connect to sia753db user sadmin using sadmin

Here you can see connection succeed

25/59 [author] Christophe Menichetti

Page 26: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Select all records from EIM table

You can verify the batch number (=100) and the batch status (=exported)

26/59 [author] Christophe Menichetti

Page 27: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

27/59 [author] Christophe Menichetti

Page 28: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

EIM Import

EIM Import Overview

Importing data into Siebel base tables is a multi step process that requires significant effort. You must first load data from an external database into the EIM tables. Subsequently, you need to run an EIM process to read the data in these EIM tables and import them into the appropriate Siebel base tables.

To import tables of data, EIM performs a sequence of tasks. Each task involves multiple passes; at least one pass is required for each EIM table included in the process. Depending on the type of import process, EIM may repeat several tasks.

The greatest difficulty of EIM import process is obviously data mapping between external data source (your data) and Siebel tables. Because of huge and complex data model of Siebel, because of complex Siebel Tables Management processes (a lot of child-parent relationship for example), it is a so hard exercise to identify which EIM columns to fill in, which EIM columns are useful and which one are no useful, which EIM columns point to which Siebel Base Table Columns.

EIM Import Process

First step: Prerequisites

That is for all these reasons described above why we are starting from data coming from export as a template for import. We are using a copy of row previously exported (with description ‘ffr ffr’) and we are going to try to import it with some modifications.

First we identify our template row (with description ‘ffr ffr’):

Select * from SIEBEL.EIM_ACTIVITY where ACT_NAME='ffr ffr'

28/59 [author] Christophe Menichetti

Page 29: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Then, to avoid the difficulty of data mapping we copied this record with some modifications.

We insert a new record into EIM_ACTIVITY table with the 4 mandatory attributes. To determine mandatory attributes for an EIM table, we can use Siebel Tools.

29/59 [author] Christophe Menichetti

Page 30: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

You obtain the list of mandatory attributes

30/59 [author] Christophe Menichetti

Page 31: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Insert the new record:

insert into SIEBEL.EIM_ACTIVITY (ACT_ACTIVITY_UID,IF_ROW_BATCH_NUM,IF_ROW_STAT,ROW_ID) values ( '1',10,'FOR_IMPORT','1' )

You can see first line for import, which is a copy of the second line above, which is exported

To finish you complete some of the other fields of record for import, following values for the same fields of exported record.

update SIEBEL.EIM_ACTIVITY set ACT_ALARM_FLAG='N' where ROW_ID='1'

31/59 [author] Christophe Menichetti

Page 32: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

You can continue to copy the exported record, as well as you want.

32/59 [author] Christophe Menichetti

Page 33: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Second step: Editing the Configuration file

Parameters in the process section apply only to that specific process and override any corresponding value in the header section for the specific process. This section describes the parameters used in the process section that are specific to an import process.

Possible parameters are the following:

33/59 [author] Christophe Menichetti

Page 34: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

NOTE: The ONLY BASE TABLES IGNORE BASE TABLES, ONLY BASE COLUMNS, and IGNORE BASE COLUMNS parameters can be used to improve EIM performance.

We use a different batch number for import (=10 for example) than this one uses for export (=100). Thus, the new configuration file for import will be the following:

[Siebel Interface Manager] USER NAME = "SADMIN" PASSWORD = "SADMIN" PROCESS = import EIM activities test #1

[import EIM activities test #1] TYPE = IMPORT // type is now import BATCH = 10 // we changed the batch number according to the value in TABLE = EIM_ACTIVITY // DB2 record

34/59 [author] Christophe Menichetti

Page 35: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Third step: Running EIM import process

You can run an import process because you have:

■ identified the data for import processing■ prepared the related EIM tables■ modified the EIM configuration file accordingly

The last think we should change, is the description field of the record we want to import (in order to improve checking result after processing). This field corresponds to EIM Column ACT_NAME as we could see previously.

You are advised to write a personal comment in order to check easier later.

update SIEBEL.EIM_ACTIVITY set ACT_NAME='my first record imported by EIM process' where ROW_ID='1'

We use command line and Siebel Server manager.

1. Put into SIEBELHOME/siebsrvr/admin the configuration file (named here testExportActivities.ifb)

35/59 [author] Christophe Menichetti

Page 36: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

2. Start the server manager in command line (command srvrmgr)

[For information on srvrmgr program, see Siebel System Administration Guide.]

srvrmgr /g localhost /e sia753_db2 /s part1_db2 /u sadmin /p sadmin

3. Run the following command to execute the EIM task

Run task for comp EIM with config= testExportActivities.ifb, errorFlags=1, SQLFlags=8

The enterprise component is EIM and the task is launched with following configuration:

testExportActivities.ifb => the configuration file errorFlags=1 => records details regarding failed rows SQLFlags=8 => records summary SQL

4. A message with completed status will be displayed

36/59 [author] Christophe Menichetti

Page 37: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

37/59 [author] Christophe Menichetti

Page 38: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Fourth step: Checking results

When an export process ends, you should carefully check the results to verify that data was successfully imported. During each import process, EIM writes comprehensive status and diagnostic information to several destinations.

First, look at Siebel application to see new record. It is the best way to see whether import process worked.

Then, query the appropriate EIM tables for rows whose IF_ROW_BATCH_NUM equals the batch number for the import.

These columns in each EIM table indicate whether a row was imported successfully, and they identify the pass number on which a row failed. During various passes of import processing, EIM sets the IF_ROW_STAT value to one of the values shown in following table:

38/59 [author] Christophe Menichetti

Page 39: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

39/59 [author] Christophe Menichetti

Page 40: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

With our example, execute the following SQL command:

select * from SIEBEL.EIM_ACTIVITY order by MS_IDENT

The next step of check of results is to read log files.

40/59 [author] Christophe Menichetti

Page 41: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

EIM is designed to import large volumes of data.

Most failures are caused by data errors. It is usually faster and easier to correct the data errors and resubmit the corrected rows as part of a subsequent batch than to reprocess an entire batch. EIM does not stop when failures occur.

Failures can occur at several steps during the EIM import process. Each type of failure has a different cause. Read the log files and see the causes listed in the following table:

41/59 [author] Christophe Menichetti

Page 42: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Read the EIM log files

Go to SIEBEL_HOME/siebsrvr/enterprises/’name of enterprise’/’name of siebel server’/log

List all log files

Edit EIM_*.log

In AIX environment:

42/59 [author] Christophe Menichetti

Page 43: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Fifth step: Solve problems and try again

Other big difficulty of EIM process is the parallel between the huge numbers of possible mappings between Siebel attributes and the little documentation about it.Thus you need to read log files to correct/improve EIM process and try again to see whether all stuffs are ok. Some errors causes the import failure, some one else causes just warning. Check, correct and try again.

An example of EIM log with an error, severity 7

Process [export EIM activities test #1] had 2 rows failon EIM_ACTIVITY for batch 1000 in step 4, pass 184: Failed to resolve foreign key value. (severity 7) // severity of error // Refer to the table above Interface table: EIM_ACTIVITY (EIM_ACTIVITY) ------------ RESOURCE_RES_NUM (Res Num) // source of the error Base table: S_ACT_CAL_RSRC (Calendar Resource for an Activity) -------------- RESOURCE_ID (Resource ID) // Explanation of error This is a foreign key value in the base table and the values in the interface table did not resolve to existing values. Verify that the IF columns correspondto existing base table rows. This failure caused the rows to be eliminated from further processing for this secondary base table. However, processing of the rows WILL continue for other destination base tables. SQL query to return rows, which were eliminated: SELECT ROW_ID, RESOURCE_RES_NUM FROM siebel.EIM_ACTIVITY WHERE (T_ACTCALRSR__STA = 184 AND IF_ROW_BATCH_NUM = 1000) // Rows eliminated ROW_ID RESOURCE_RES_NUM

43/59 [author] Christophe Menichetti

Page 44: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

-------- ---------------- 1 NULL 1-34SSX NULL -------- ---------------- (all 2 rows which failed)

In the previous example, we just needed to enter a value for the field ‘RESOURCE_RES_NUM’. This error caused the failure of all EIM import process.

The status was ‘PARTIALLY_IMPORTED’.

- - -

An other instance of log file is the following. Error with severity 8, which doesn’t stop the EIM process. If you want, you can fix it or not.

At the file end, you can see the error total. Here 51 errors, which are, in fact, warnings-like.

An example of EIM log with an error, severity 8 (warning)

Process [export EIM activities test #1] had 1 row fail On EIM_ACTIVITY for batch 10 in step 7, pass 196: Failed to resolve foreign key value. (severity 8) // severity 8 DOESN’T CAUSE // STOP OF PROCESSInterface table: EIM_ACTIVITY (EIM_ACTIVITY) ------------ TODO_ACTIVITY_UID (Activity User Id) Base table: S_EVT_ACT (Activity) --------- TODO_APPT_ID (Todo Appointment) This is a foreign key value in the base table and the values in the interface table did not resolve to existing values. Verify that the IF columns correspond toexisting base table rows or new interface table rows. This failure did not eliminate the rows from further processing. Processing

44/59 [author] Christophe Menichetti

Page 45: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

will continue for other columns in the same and in other destination base table. -----------------------------------------------------------------------

Recorded 51 groups of failures. // Total of errors (may be warning)

45/59 [author] Christophe Menichetti

Page 46: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

How to manage Foreign Key Mapping

What to do after?

As you can guess, it could be very boring to fix all 51 errors. All these errors refer to foreign key (pointer to another table) problem “This is a foreign key value in the base table and the values in the interface table did not resolve to existing values” and we are not obliged to solve it (you can decide to not attach a contact to an activity for example, without invalidating the activity record at application level). You have to select foreign key in which you are interested, depending on your data model or business case, or even your simple choice.

Example

Keeping the former example of errors (51 errors), we are going to show how to fix a foreign key problem.

We decided (it is our choice) to fix the following error with severity 8:

on EIM_ACTIVITY for batch 10 in step 4, pass 110: Failed to resolve foreign key value. (severity 8) // not a real error Interface table: EIM_ACTIVITY (EIM_ACTIVITY) // for all of the following attributes, values ------------ // are missed to find an existing foreign keyACT_AST_BI (Name) ACT_AST_NUM (Asset number) ACT_ASTPRDVEN_NAME (Name) ACT_AST_PRDVEN_BI (Name) ACT_AST_PRDVEN_LOC (Site) ACT_AST_PROD_BI (Name) ACT_AST_PROD_NAME (Name) ACT_REV_NUM (Rev Num) Base table: S_EVT_ACT (Activity) --------- ASSET_ID (Asset ID) // it is a foreign key pointing to another table

46/59 [author] Christophe Menichetti

Page 47: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

47/59 [author] Christophe Menichetti

Page 48: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Action

First we need to search ‘ASSET_ID’ column in S_EVT_ACT table in order to find the foreign table. Using Tools, you will see ‘ASSET_ID’ column point to S_ASSET table.

48/59 [author] Christophe Menichetti

Page 49: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Then the more difficult is to determine to which columns in ‘S_ASSET’ table, the following column list is corresponding:

ACT_AST_BI (Name) ACT_AST_NUM (Asset number) ACT_ASTPRDVEN_NAME (Name) ACT_AST_PRDVEN_BI (Name) ACT_AST_PRDVEN_LOC (Site) ACT_AST_PROD_BI (Name) ACT_AST_PROD_NAME (Name) ACT_REV_NUM (Rev Num)

Indeed, in order to correctly enter right values in EIM columns (values pointing to an existing row in ‘S_ASSET’ table) to map to a correct foreign key, we need to understand foreign key mapping:

EIM_ACTIVITY S_ASSET---------------- ---------ACT_AST_BI (Name) ????????? ACT_AST_NUM (Asset number) ????????? ACT_ASTPRDVEN_NAME (Name) ????????? ACT_AST_PRDVEN_BI (Name) ????????? ACT_AST_PRDVEN_LOC (Site) ????????? ACT_AST_PROD_BI (Name) ????????? ACT_AST_PROD_NAME (Name) ????????? ACT_REV_NUM (Rev Num) ?????????

EIM_ACTIVITY S_ASSET

EIM Load S_EVT_ACT

PointTo Foreign key

ASSET_ID

49/59 [author] Christophe Menichetti

Page 50: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Managing foreign key mapping with Siebel Tools

Siebel Tools helps you to manage the foreign key mapping, giving you explicit columns concerned by the mapping. To see such explicit mapping, let you follow the following procedure:

1- query EIM table for EIM_ACTIVITY table

2- click on EIM Table Mapping in Object Explorer

50/59 [author] Christophe Menichetti

Page 51: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

3- Select ‘S_EVT_ACT’ table in EIM table Mappings

4- Click on foreign key mapping

5- Click on ‘A’ letter on tabs

51/59 [author] Christophe Menichetti

Page 52: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

6- Select ‘ASSET_ID’ mapping name

7- Select Foreign key mapping column

52/59 [author] Christophe Menichetti

Page 53: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

These following screenshots show you the mapping between EIM columns (Interface table data column) and ‘foreign’ table S_ASSET (user key attribute)

EIM_ACTIVITY S_ASSET

To sum up, he screenshot above give you the following mapping:

Take a glance at user key attribute column. This column indicates you the exact value to enter in corresponding interface column.

Since a foreign key in a foreign table can point itself to another foreign key and so on, the search of the right value to enter in foreign key mapping could be so fastidious. The following section explains you how to resolve foreign key mappings.

53/59 [author] Christophe Menichetti

Page 54: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

Resolving foreign key mappings

Using the highlighted row example above, we are going to expose the different step to resolve a foreign key mapping value.

What does it mean? :ACT_AST_PRDVEN_BI PROD_ID/VENR_OU_ID/BU_ID/NAME

1. Go to S_ASSET Table

2. Navigate to user key in S_ASSET table

3. Select traditional user key

54/59 [author] Christophe Menichetti

Page 55: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

4. Navigate to user key attribute in user key S_ASSET_U1

5. Select our user key attribute example

PROD_ID/VENR_OU_ID/BU_ID/NAME

55/59 [author] Christophe Menichetti

Page 56: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

6. Navigate to user key attribute join

7. Here you will find a path of jointure

56/59 [author] Christophe Menichetti

Page 57: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

The join path must be read from the greatest join sequence number to the lowest join sequence number.

TIP: you find again this order in the name of the user key attribute

PROD_ID / VENR_OU_ID / BU_ID /NAME

3 2 1 0

This sequence means PROD_ID column in S_ASSET table is a foreign key pointing to S_PROD_INT table.

In this S_PROD_INT table, we must consider VENDR_OU_ID column, which is also a foreign key pointing to S_ORG_EXT table.

In this S_ORG_EXT table, we must consider BU_ID column, which is also a foreign key pointing to S_BU table.

Finally, in this S_BU table, we must consider NAME column.

57/59 [author] Christophe Menichetti

Page 58: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

But ACT_AST_PRDVEN_BI column must contain row_id value and not a name value.

There is a very specific Siebel way to populate such ‘_BI’ column (organization column): you must populate the equivalent ‘_BU’ column with the name (here it will be ACT_AST_PRDVEN_BU).

You can find the explanation in the next section.

Result:We must fill in EIM interface column ACT_AST_PRDVEN_BU

with thecorresponding value of S_BU.NAME

58/59 [author] Christophe Menichetti

Page 59: EIM Load Explanation With an Example

Europe Oracle|Siebel IBM Support Center Montpellier – PSSC

IMPORTANT NOTE: organization columns

The EIM_ type interface tables use the xxx_BU/xxx_BI column pairs to map organizations. For example, the CON_BU/CON_BI column in the EIM_CONTACT interface table is mapped to the BU_ID column in the S_CONTACT base table.In order for organizations to be resolved properly, you need to populate the xxx_BU column with the organization name and leave the xxx_BI column empty.

Do not populate the xxx_BU column with the organization ROW_ID. EIM looks up the ROW_ID for the organization in xxx_BU and puts it in the corresponding xxx_BI column!!!

59/59 [author] Christophe Menichetti