Pervasive Tutorial

Integration Architect

Tutorials ManualTutorials for Map Designer, Structured Schema Designer

and Process Designer

Pervasive Software Inc.12365 Riata Trace Parkway

Building BAustin, TX 78727 USA

Telephone: 512 231 6000 or 800 287 4383Fax: 512 231 6010

Email: [email protected]: http://www.pervasive.com

R e f e r t o t h e l i c e n s e . t x t f i l e i n t h e d e f a u l t i n s t a l l a t i o n d i r e c t o r y f o r d i s c l a i m e r s a n d i n f o r m a t i o n a b o u t t r a d e m a r k s a n d c r e d i t s .Tutorials ManualApril 2005

Contents

Contents1 Map Designer Tutorials . . . . . . . . . . . . . . . . . . . . . . . . 1-1

Step-by-Step Tutorials for Learning Map Designer and Structured Schema Designer

Introduction to Map Designer/Structured Schema Designer Tutorials . . . . . . . . . 1-2Map Designer Tutorial 1 - The Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3

Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-7

Map Designer Tutorial 2 - Parsing an ASCII File. . . . . . . . . . . . . . . . . . . . . . 1-8Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-8Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-8Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-8Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-13

Map Designer Tutorial 3 - Output to an Existing Target File . . . . . . . . . . . . . . . 1-14Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-14Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-14Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-14Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-17

Structured Schema Designer Tutorial 4 - Using the Data Parser . . . . . . . . . . . . . 1-18Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-18Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-18Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-18Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-29

Map Designer Tutorial 5 - Single-record Type File to a Multi-record Type XML File . 1-30Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-30Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-30Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-30Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-40

2 Process Designer Tutorials . . . . . . . . . . . . . . . . . . . . . . 2-1Step-by-Step Tutorials for Learning Process Designer

Introduction to Process Designer Tutorials. . . . . . . . . . . . . . . . . . . . . . . . . 2-2Process Designer Tutorial 1 - The Basics . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3

Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9

Process Designer Tutorial 2 - Loops and Queues. . . . . . . . . . . . . . . . . . . . . . 2-11Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-11Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-11

1

Contents

Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-11Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-19

3 Extract Schema Designer Tutorials . . . . . . . . . . . . . . . . . . 3-1Step by Step Tutorials for Learning Extract Schema Designer

Introduction to Extract Schema Designer Tutorials. . . . . . . . . . . . . . . . . . . . 3-2User Driven Tutorials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2

Tutorial 1 - The Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4Tutorial 2 - Tagged Data and Automatic Features. . . . . . . . . . . . . . . . . . . . . 3-11Tutorial 3 - Columnar Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-20Tutorial 4 - Floating Tags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-29Tutorial 5 - Columnar Data with a Footer . . . . . . . . . . . . . . . . . . . . . . . . . 3-37Tutorial 6 - Variable Length Multi Line Data Fields . . . . . . . . . . . . . . . . . . . 3-45Tutorial 7 - Multiple Accept Records. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-51

2

c h a p t e r

1
Map Designer Tutorials
Step-by-Step Tutorials for Learning Map Designer and Structured Schema Designer

This chapter includes five tutorials to help you learn Map Designer and Structured Schema Designer. Tutorials 1, 2, 3 and 5 guide you through creating Transformations and Maps in Map Designer. Tutorial 4 walks you through using the Data Parser in Structured Schema Designer.

This chapter includes the following sections:

“Introduction to Map Designer/Structured Schema Designer Tutorials” on page 1-2

“Map Designer Tutorial 1 - The Basics” on page 1-3

“Map Designer Tutorial 2 - Parsing an ASCII File” on page 1-8

“Map Designer Tutorial 3 - Output to an Existing Target File” on page 1-14

“Structured Schema Designer Tutorial 4 - Using the Data Parser” on page 1-18

“Map Designer Tutorial 5 - Single-record Type File to a Multi-record Type XML File” on page 1-30

1-1


Introduction to Map Designer/Structured Schema Designer Tutorials

Five step-by-step tutorials are available to help you learn how to use Map Designer and Structured Schema Designer. The tutorials should be completed in order. Each Tutorial builds upon your knowledge as you work through them. Also, you must run Tutorial 1 to create a Target file that will be used in Tutorial 3.

“Map Designer Tutorial 1 - The Basics” on page 1-3

“Map Designer Tutorial 2 - Parsing an ASCII File” on page 1-8

“Map Designer Tutorial 3 - Output to an Existing Target File” on page 1-14

“Structured Schema Designer Tutorial 4 - Using the Data Parser” on page 1-18

“Map Designer Tutorial 5 - Single-record Type File to a Multi-record Type XML File” on page 1-30

1-2

Map Designer Tutorial 1 - The Basics

Map Designer Tutorial 1 - The BasicsThe purpose of Tutorial 1 is to guide a new user through the basic steps necessary to create, save, and run a basic data Transformation Map in Map Designer. It will help you to become familiar with both the basic process of converting data from one application to another using Map Designer and the terms used throughout the documentation. The best way to use this tutorial is to print a hard copy so you can follow the sequential steps.

Note The Target file created in Tutorial 1 will be used as the Target file in Tutorial 3, so you must run this tutorial before running “Map Designer Tutorial 3 - Output to an Existing Target File” on page 1-14.

Goals In this tutorial, you will:

learn the basic process of data conversion,

design a simple data Transformation Map,

save and run the Map, and review the Target file, and

learn new terms you will encounter throughout the documentation.

Requirement Create, save, and run a basic data Transformation Map.

Procedure 1 From Repository Explorer, click the down arrow beside the New icon in the toolbar and select Map to open the Map Designer window.

2 Click the down arrow beside the Source Connection box to open the Select Connection Window.

You will have a choice of three tabs in the Select Connection window: Most Recently Used, Factory Connections and User Defined Connections.

3 Click the Factory Connections tab and click ASCII (Delimited) from the list. Click OK to return to the Map Designer window.

4 Click the down arrow next to the Source File/URI box.

1-3


This will open the standard Windows navigation dialog box. The source file you will be using, Tutor1.asc, in the default installation directory \Common800.

Note The default file extensions for ASCII delimited are .asc, .csv and .txt.

5 Click Tutor1.asc and then click Open.

You will be taken back to the Source Connection tab and the drive, directory and filename will be visible in the Source File box.

6 Click the Source Data Browser icon to display the source file. You will see that it contains field names in the first row of data. This is the file header. You will need to change the default source properties so that Map Designer will recognize this. Close the Source Data Browser by clicking on the X button in the top right.

The Source Properties options box is on the right side of the Source tab. This is where specific properties of the Source file can be set.

7 Click False next to the word Header in the Source Property options box, and then double-click to change it to True.

8 Click Apply below the list of Property options.

9 Click the Target Connection tab.

10 Click the down arrow next to the Target Connection box to open the Select Connection window.

You will have a choice of three tabs in the Select Connection window: Most Recently Used, Factory Connections and User Defined Connections.

11 Select the Factory Connections tab.

12 Select dBASE IV from the list. Click OK to return to the Map Designer window.

1-4


13 Click the down arrow to the right of the Target File/URI box. This opens the standard Windows navigation dialog box. The default file extension will appear in the File Name box (in this case, ".dbf "). Replace the asterisk (*) with the file name you want to use and click Open.

Name the file Tutor1.dbf (".dbf " is the standard extension for dBASE files), since this file name will be used in Tutorial 3.

14 Click Connect , if it is not disabled.

15 The Output Mode will default to Replace File/Table when you are creating a new Target file. This output mode will create a new file or replace any existing file of this name in this directory.

16 Click the Map tab.

17 The Map tab is divided into four panes. The upper left pane shows the record types and the Event Handlers, Fields and Rules associated with each record type. The tree will be expanded and the fields for record type R1 highlighted when you open the Map tab.

18 The upper right pane contains a grid showing the fields details for the selected record type.

19 In the upper right grid, locate the box that contains an asterisk (*) next to the cell that says <all fields>. This asterisk represents the layout of all fields in the Source File and acts as a shortcut to duplicate the entire Source Schema in the Target file. Click this cell and hold down the mouse button.

20 Drag the cursor down to the lower grid. The cursor will change to a black rectangle when it is over a cell in which the selected file information can be inserted. Release the mouse button when the cursor is in the first cell of the lower grid.

21 Since dBASE does not allow lowercase letters or spaces in field names, an Invalid Field Name dialog box will appear. Map Designer will offer to fix the problem in various ways. The default is to automatically create valid field names based on the Source field name for all invalid fields.

22 Click the OK button on the Invalid Field Names dialog box.

1-5


23 After you click the OK button, all the lowercase letters will change to uppercase and spaces will change to underscores for the Target field names. The Target Field Expression column will be filled with the Source field names in appropriate notation for mapping expressions.

24 In the upper left pane of the Map tab, navigate to the AfterEveryRecord event handler by expanding Record Types, R1 and R1 Event Handlers. Select AfterEveryRecord.

25 This will cause the Action Parameter grid to appear.

26 Click in the Action column. An ellipsis button will appear. Click this to display the Event Action definition box. Click the down arrow under the Action Name box and choose ClearMapPutRecord from the drop down list.

27 ClearMapPutRecord is the most commonly used action in a transformation. If you forget to add it in an ordinary single record type to single record type transformation, it will be added for you automatically.

28 Since there is only one record type in the Target, the default selection of Target record type R1 will be the correct one. The use of the other parameters in this Event Action and the use of all the other Event Actions are explained elsewhere in the Help.

29 Click OK.

30 You have now completed the three transformation steps.

31 Click the Save Map icon in the toolbar.

32 This will open the Save As dialog box in which you are prompted for a Save Location. This will default to your Workspace, however you should verify the location prior to saving the map and change it if needed. Maps default to the document type ".map.xml".

33 Type a document name and click Save. Your transformation is saved.

34 Click the Run Map icon in the toolbar.

1-6


Map Designer will load all the information it needs to run this transformation and then execute it. You will see a gauge quickly flash on the screen as the transformation progresses. A counter will estimate the number of records converted. When the transformation is finished, the total number of records read and the number converted will be displayed in the status bar.

35 Click the Target Data Browser icon in the tool bar to view the new dBASE IV file.

This will open a new window in which you can view and browse the dBASE IV file just created.

36 After viewing the dBASE file, close the browser by clicking the X button in the upper right corner.

37 Close Map Designer by clicking the X in the upper right corner.

You will be back in the Repository Explorer, where you will see the Map you just created in the Repository Documents list in the right panel. If you do not see the new Map, click on your workspace repository in the left panel tree structure.

Congratulations, you have just created, saved and run your first data Transformation Map using Map Designer!

Summary In this tutorial, you:

learned the basic process of data conversion,

designed a simple data Transformation Map,

saved the Map, ran it, and reviewed the Target file, and

learned new terms you will encounter throughout the documentation.

1-7


Map Designer Tutorial 2 - Parsing an ASCII FileTutorial 2 is designed to show you how to apply an existing Structured Schema to a Source file that is not structured.

In this tutorial you will convert the data in a Fixed ASCII file to a Lotus 1-2-3 r4 file. By following the steps outlined below, you will become familiar with using a Structured Schema (formerly known as a record layout or data structure), transferring data from one file format to another and the terms used throughout Map Designer documentation.

Goals In this Tutorial, you will:

convert data in an ASCII Fixed file to a Lotus 1-2-3 r4 format,

use a Structured Schema.

Requirement Use a Structured Schema in the transformation of an unstructured Source file.

Procedure 1 In the Repository Explorer, open the Map Designer window by doing the following: In the toolbar, click the down arrow beside the New icon, then select Map.

2 Click the down arrow to the right of the Source Connection box. This opens the Select Connection window.

3 You will have a choice of three tabs in the Select Connection window: Most Recently Used, Factory Connections and User Defined Connections.

4 Click the Factory Connections tab and select ASCII (Fixed) from the list. Click OK to return to the Map Designer window.

5 Open the Windows navigation dialog box by clicking the down arrow next to the Source File/URI box.

Note The file extensions for fixed ASCII files are .asc, .csv and .txt.

Tutor2.asc will be found in the default installation directory …\Common800.

6 Select Tutor2.asc and then click the Open button.

The Source Connection tab is redisplayed with the selected file and its path displayed in the Source File box.

1-8

Map Designer Tutorial 2 - Parsing an ASCII File

7 Click the Source Data Browser icon in the toolbar.

You will see Tutor2.asc in its unparsed format (shown next).

8 Close the browser window by clicking the X button on the upper right corner.

9 Open the Select Structured Schema Document window by clicking the down arrow to the right of the Structured Schema box.

Tip A Structured Schema file for Tutor2 has been provided for you, and is called Tutor2.ss.xml. The file can be found in the Program Files … \Common800 directory (this is your default installation directory). You must copy this file and paste it into your default Workspace. For instance, copy the Tutor2.ss.xml file and paste it into the following sample location:

C:\MyWork\User_Name\Workspace1\xmldb

10 Click the Tutor2.ss.xml file and then click OK.

11 Click the Source Data Browser icon on the toolbar to view the parsed file. Notice the data is now separated into field columns (as shown next).

1-9


You can jump to any record in the Source file by typing its record number in the Record No box in the lower right portion of the window. You can also change how the data is displayed by clicking the down arrow to the right of the Character Display Mode box. For information about the various Display Mode options, see “View Source Data” on page 8-17 in the Map Designer User’s Guide.

12 Close the Source Browser by clicking the X in the upper right.

13 Click the Target Connection tab to choose your Target Connection and Target File.

14 Click the down arrow next to the Target Connection box to open the Select Connection Window.

You will have a choice of three tabs in the Select Connection Window: Most Recently Used, Factory Connections and User Defined Connections.

15 Click the Factory Connections tab. Select Lotus 1-2-3 r4 from the list and click OK. Notice that because Lotus 1-2-3 r4 is the Target type, the default file name is "*.wk4".

16 Enter a Target File name and click Open. In this tutorial we will call the file Tutor2.tf.wk4.

Tip Always use consistent naming conventions when creating target file and maps.

17 Click Connect, if the button is not disabled.

1-10


18 The Output Mode will default to Replace File/Table when you are creating a new Target file. This output mode will create a new file or replace an existing file with this name.

Now that you have defined your Source and defined your Target, you now need to set up the Target layout. This is done in the Map tab.


The Map tab is divided into four panes. The top and bottom left panes show the Source file and Target file tree structures. These show the record types and the Event Handlers, Fields and Rules associated with each record type.

The upper right pane contains a grid showing the fields details for the Source record type. The field names, sizes and types are taken from the Structured Schema you attached.

The lower right pane contains a grid showing the Target record layout. The Target record layout should be empty. All things associated with the Target will appear in the grid that matches the Target tab color.

Note If the Target grid is not empty, then someone has previously completed this tutorial and Map Designer is displaying the structure of the file that was previously created. If so, click the Target Field Names heading above the first column in the Target layout grid. This will select the entire layout. Then use the <DELETE> key on the keyboard to delete the layout.

20 To use headers in your Target file, click the Target Connector Properties icon in the View menu. Change the HeaderRecordRow property from 0 to 1. Click OK to save your changes and close the dialog.

21 In the Source grid, move the cursor to the cell before the cell that says <all fields>. This cell contains an asterisk, which represents the layout of all fields in the Source file and provides a shortcut to duplicating the entire record structure to the Target file. Click and hold the mouse button on this cell.

1-11


22 Without releasing the mouse button, drag the cursor down to the first cell in the Target Field Name column of the lower right grid. The cursor changes to a black rectangle indicating that this is a valid position for it to be released. Release the mouse button.

This will copy all the Source fields into the Target grid in the same order. Notice the Target Field Expression cell for each field is also filled.

In a simple drag-and-drop procedure, you have set up the data structure of your Target file and completed your mapping.

23 Click the Save Map icon in the tool bar.

This will open the Save As dialog box in which you are prompted for a Save Location. (This should default to your Workspace. However, verify the location prior to saving the document and change to your Workspace if needed.) Maps default to a document type of .map.xml. Type a document name and click Save.

24 Click the Run Map icon in the tool bar.

Note A box may appear when you try to run this map. It will say, "No event handlers were defined for the map". Event handlers are used to control the transformation process, including the way data is written to the Target. Because this is a simple transformation of a single record type to a single record type without any manipulation of fields, a default event handler will be added to this transformation. Click OK to continue.

Map Designer will now execute the map, reading data from your Source file, transforming it and exporting it to your Target file. When the transformation is finished, the total number of records read and the number converted will be displayed in the status bar.

25 Click the Target Data Browser icon in the tool bar to view the Lotus 1-2-3 r4 file. This allows you to review the results of any expressions and transformations applied to the data.

26 Close the Target Data Browser by clicking the X button in the upper right corner.

1-12


27 Close Map Designer by clicking the X button in the upper right corner to return to the Repository Explorer window.

You will see the Map you just created on the list, along with information about the Map.

Congratulations, you have now used a Structured Schema in Map Designer!

Summary In this Tutorial, you:

converted data contained in an ASCII Fixed file to a Lotus 1-2-3 r4 format,

learned how to use a Structured Schema.

1-13


Map Designer Tutorial 3 - Output to an Existing Target FileTutorial 3 demonstrates how to create, save, and run a data Transformation Map to an existing Target file. You will overwrite the Target file new data in the same structure as the existing Target.

Note If you have not already completed Tutorial 1, please do so now. The Target file created in Tutorial 1, Tutor1.dbf, will also be the Target file in this tutorial.


transfer data from one application file format to another,

overwrite data in an existing Target file, and

learn terms that are used throughout the Map Designer documentation.

Requirement Transform the data in an ASCII Delimited file to an existing dBASE IV file.

Procedure 1 From the Repository Explorer, click the down arrow beside the New icon in the toolbar and select Map to open the Map Designer window.



3 Click the Factory Connections tab and select ASCII (Delimited) from the list. Click OK.

4 Click the down arrow next to the Source File/URI box to open the standard Windows navigation dialog box.

Note The default file extensions for ASCII delimited are .asc, .csv and .txt.

1-14

Map Designer Tutorial 3 - Output to an Existing Target File

5 Select Tutor3.asc and then click Open.

Since the Delimited ASCII Source file has a header record that you want to be converted to field names in your Target file, you need to update the Source Properties on the right side of the Source tab.

6 Move the cursor to the value cell for Header and change to True.

7 Click Apply below the list of property options.

8 Click the Target Connection tab.

9 Click the down arrow next to the Target Connection box to open the Select Connection Window.

You will have a choice of three tabs in the Select Connection Window—Most Recently Used, Factory Connections and User Defined Connections.

10 Click the Factory Connections tab.

11 Click dBASE IV from the list and click OK.

12 Click the down arrow next to the Target File/URI box. The Windows navigation box is display. Notice that because dBASE IV is the Target type, the default file name is "*.dbf ".

Instead of creating a new file, as you did in the previous tutorials, you want to overwrite the data in the existing file: "Tutor1.dbf ".

13 Select Tutor1.dbf and click Open.

Locate the Output Mode box at the bottom of the Target tab. Notice the setting is Append to File/Table. The Output Mode defaults to Append to File/Table when you choose an existing file as your Target file. For this tutorial, you want to overwrite the data in the "Tutor1.dbf " file, so you must change the Output Mode from Append to File/Table to Replace to File/Table.

14 Click the down arrow to the right of the Output Mode box and click Replace File/Table.

15 Click the Target Schema icon in the tool bar. In the Target Schema window, you will see that the field names, types and lengths are already shown. By choosing an existing data file as your Target file, Map Designer reads the structure of that file and displays it in the Target Schema window.

1-15


The goal in this tutorial is to overwrite the data while keeping the same structure of the Target data file.

16 Close the Target Schema Window by clicking the X button in the upper right.


You will see there is field information in both the upper right Source and the lower right Target grids. However, the Target Field Expression cells are all empty. To map data from the Source data file to the correct fields in the Target data file, you must tell Map Designer which data fields go where.

18 Right-click anywhere in the lower right grid and choose the Match by Name option. Since most of the field names in the Source file match the field names in the Target file, most of the mapping work will be done. However, as the Last Pymnt in the Source is different to PAYMENT in the Target, the Target Field Expression remains blank for this field in the lower grid.

19 In the lower right Target grid, move the cursor to the Target Field Expression cell for the PAYMENT field and click once. Make sure the cursor is to the right of the equal sign ( =). Click the down arrow on the right to display the list of Source fields. Click Records("R1").Fields("Last Pymt") to select this field.

20 By using the mapping process, you have instructed Map Designer to transfer the data from specific fields in the Source file to specific fields in the Target file, including a field that has a different field name. This allows you to transfer data from any field in a Source file to any field in a Target file.

21 You have now specified your Source Connector type, Source File, Target Connector type and Target File. You also have created a Target structure that includes some mapping. This is enough information for Map Designer to transform the data.

22 Click the Save Map icon in the tool bar.

23 This will open the Save As dialog box. Type a document name and click Save.

24 Your Map is saved.


1-16

Map Designer Tutorial 3 - Output to an Existing Target File

Note A box may be displayed at this point containing the message: "No event handlers were defined for the transformation". Event handlers are used to control the transformation process, including how data is written to the Target. As you have created a simple single record to single record transformation, a default event handler will be added automatically to this transformation. Click OK to continue.

26 A Windows alert box will appear containing the message: "Tutor1.dbf exists and will be overwritten. Do you want to overwrite this file?" Click Yes.

Map Designer will then load all the information it needs to run this transformation and execute it. When the transformation is finished, the total number of records read and the number converted will be displayed in the status bar.

27 After the transformation is complete, you can view the new dBASE IV file.

28 Click the Target Data Browser icon in the tool bar to view the new dBASE IV file. This will open a new window in which you can browse the dBASE IV file that was just created.

29 To close the Target Data Browser, click the X at the upper right corner.

30 Close Map Designer by clicking the X button in the upper right corner to return to the Repository Explorer window, where you will see the Map you just created on the list.

Congratulations, you have just created, saved and run a Map in Map Designer!

Summary In this Tutorial, you . . .

transferred data from one application file format to another,

overwrite data in an existing Target file,

preserved the structure of the Target file, and

learned terms that are used throughout the Map Designer documentation.

1-17


Structured Schema Designer Tutorial 4 - Using the Data Parser

Tutorial 4 is designed to demonstrate how to use the Data Parser feature in Map Designer. The Data Parser enables you to create a structure for a file that does not have an internally stored data structure. ASCII Fixed, binary, and C-Tree are three examples.

The best way to use this tutorial is to print it out so you can refer to it easily as you go along. Also, the last section of this tutorial shows the layout of the sample binary file tutor4.bin.


learn how to navigate Data Parser, and

learn how to use Data Parser to create a structure for unstructured data.

Requirement Use the Data Parser to create a structure for an unstructured Binary Source file.

Procedure 1 From the Repository Explorer, click the down arrow beside the New icon in the toolbar and select Structured Schema to display the Structured Schema Designer window.

2 At the top of the window, middle box, click the drop down arrow and select Binary as the Connector type.

1-18


3 In the tool bar, click the Parse Data icon. The Enter Connection Information window is displayed.

4 At the Source File/URI box, click the drop down arrow and select the Tutor4.bin file. Click Open.

5 The sample file has a small header of 35 bytes of data; it is not part of the records and must be ignored. So find StartOffset in Connector Properties (right half of window), enter 35 and click Apply. Click OK to close the window.

In the graphic below, you will now see the start of your Source file in the Parser window. The records are currently 100 bytes long and the records are not lined up yet.

1-19


6 Set a known record length in the Length box(top center of the window). Enter 67 and press <ENTER>.

Caution You now have to mark the fields within the record. It is important to define the fields from left to right. If you define a field size incorrectly, it will throw off the definitions for every field afterwards.

1-20


7 Click the last character of the first field in the top row of actual data. This is the pale blue row beneath the yellow ruler. In this example this is the final "s" in “Us” of Conversions R Us. A field marker (a double-ended arrow) will be displayed at this position in the yellow area beneath the ruler.

In this case, it is easy to see where the first field ends because it is an ordinary text field and ends right where an incremental numeric data field starts.

8 See the file record layout at the end of this tutorial. It provides the field type, size, name, etc. needed to define the fields. The record layout for Field 1 shows Size 16. Click the ruler at the 16th hash mark and the field marker will be displayed. To delete a marker, click on the ruler hash mark and the marker will disappear. Then create a new marker in the correct position.

9 Beneath the data is a box labeled Field Name. Highlight the default field name, Field1 and overtype with a new field name. In this case, enter Company.

10 To set the data type for this field, click the drop down arrow (right of the Data Type box). In this case, the data type is Text, the default.

11 Set the Property values for Company (lower right of the window). Each datatype has its own specific properties. For this Text field, nothing needs to be changed.

12 Click in the yellow area directly above the field and the Contents box (bottom left) will show the data for that field formatted according to the Length, Data Type and Properties you have just set up. Do not to click in the blue area or you will create a field marker and will have to remove it.

13 For Field 2, count four positions on the ruler; click the mouse to mark the end of Field 2 and the beginning of the next. A marker will be displayed at the 20th position and the Property Size value will display "4". Rename the field RecordNumber.

14 Select the Data Type for this field. In this case it is Zoned decimal. Next to the Data Type box, click the down arrow and select Zoned decimal.

Note that the Offset box in the lower left shows the starting position of each field. This value is for display only and cannot be changed.

1-21


15 Enter the Property values for this field. This field is not signed, so in the Signed box, click the down arrow and select No. You do not want to display figures after the decimal point for this field so in the box labeled Decimals, highlight type 0. Press <ENTER> as the rest of the properties are already set to their correct values.

16 Click in the yellow area directly above this field and the Contents box will show the first record’s data for this field formatted according to the field Length, Data Type and Property values you have set up.

17 Count four hash marks to the right of the last field’s arrow and click your mouse on the ruler. Rename Field3 as DegreeOfVar. Select Type as 32-bit IEEE floating-point. Under Property, scroll down to Decimals and type 3. Leave the other properties as their defaults. Look in the Contents box. You should see 35.957. If you do not, see the Contents box note below.

Note The Contents box is the best visual cue when trying to set the data type and properties of a field correctly. If you know what the field should contain, you will be able to judge by the Contents box display if you have set the field length, data type and data properties correctly. When you parse a file, you may need to go through the tedious process of choosing each field's length, type and properties by trial and error. ** The Contents box display will default to showing the contents of the first field whenever you change anything; you will need to click the hash mark in the ruler each time you want to show what it contains.

18 Use the “Record Layout” on page 1-25 to continue defining the rest of the fields. Use the following steps:

Set field length with the mouse.

Rename the field.

Select the data type.

Set the property values.

After checking the Contents box, click OK when you are satisfied with the results of each field.

19 When you are finished parsing the file, the Parser display should look like the following screenshot:

1-22


Last Field: Do not try to mark the end of the last field, as this is also the end of the record. Instead, in the Field Name drop down list, choose Serial Number and make sure the Size is 9.

20 Read and follow the Caution note below.

Caution When you are finished parsing the data, you must save your changes, as they are not automatically saved as you go along. If you close the Parser without saving, you will have to start over. In the toolbar, click Incremental Save and then OK to close the Parser.

21 Click Save Structured Schema.

The Save Structured Schema Document As window will open. In the Document Name box, at the bottom of the window, type Tutor4. Your Schema will be saved as an .ss.xml file. Click Save.

1-23


22 You may want to add a Description of this Structured Schema to help you identify its intended purpose later. To add a description, click the View menu in the Structured Schema Designer window, and select Properties. This window will display the Author Name if you need to change it, and a box to enter the Description. Click OK after you have made any changes. If you enter a description after saving the Schema, you will have to Save once more to save the Description.

Congratulations! You have created a Structured Schema. Now it is time to use it in a Map in the Map Designer. Click the X in the upper right corner of the Structured Schema Designer window to return to the Repository Explorer.

To Use a Structured Schema1 From the Repository Explorer, click the down arrow beside

the New icon in the toolbar; select Map to open the Map Designer.

2 Next to the Source Connection, click the drop down arrow to open the Select Connection window. You have a choice of three tabs: Most Recently Used, Factory Connections and User Defined Connections. Click the Factory Connections tab and select Binary from the list. Click OK.

3 Click the down arrow to the right of the Structured Schema box. Select the file you created earlier in this Tutorial, Tutor4.ss.xml, and click Open.

4 Click the down arrow next to the Source File/URI box. Select Tutor4.bin and click Open.

5 Set the Starting Offset. In the Properties section, find the Start Offset value, and overtype with the starting offset value: 35. Click the Apply button below the list of property options.

6 Click the Source Data Browser icon on the toolbar to view the parsed file. Notice the data is now separated into field columns (as shown next).

1-24


7 You can display any record in the Source file by typing the record number in the Record No box in the lower right of the window. You can also change how the data is displayed by clicking the down arrow to the right of the Character Display Mode box. For information about the various Display Mode options, see “View Source Data” on page 8-17 in the Map Designer User’s Guide.

8 Close the Source Browser by clicking the X button in the upper right corner.

9 Click the Save Map icon in the tool bar. This will open the Save As dialog. Enter a document name and click the Save button. Your Map is saved.

Record Layout

HEADER: 35 Bytes

RECORD LENGTH: 67 Bytes

Field Name Type Size Content of First Record

Other Properties

1 Company text 16 Conversions R Us

(None)

2 RecordNumber zoned decimal 4 1 Precision: 4 Signed: No Sign Pos: Trailing Style Microfocus COBOLScale: 0 Decimals: 0

3 DegreeOfVar 32-bit IEEE floating-point

4 35.957 Precision: 7Signed: Yes MSB First: No Decimals: 3

1-25


How to determine starting offset and record lengthOften if you need to create a structured schema for a file without a layout, you will have to determine the starting offset and record length. It is helpful if you have a printout of the first few records from the file to hand so that you can compare the formatted contents with the actual data held in the file.

The following steps show how to determine the starting offset and record length:

1 From the Repository Explorer, click the down arrow beside the New icon in the toolbar. Then select Structured Schema to bring up the Structured Schema Designer window.

2 Click the down arrow next to the Connector type box (the middle box at the top of the window). Select the connector type from the drop down list.

3 Note that if the Parse Data icon becomes grayed out after you have selected a connector type, then the connector type chosen does is not one that needs to be parsed.

4 Bytes zoned decimal 9 3456680 Precision: 9Signed: No Sign Pos: Trailing Style: Microfocus COBOL Scale: 0 Decimals: 0

5 ErrOccur Boolean 1 True Picture: True.False

6 Reversal display sign trailing

10 -99756 Precision: 10Signed: Yes Sign Pos: Trailing Style: MicroFocus COBOL Scale: 0 Decimals: 0

7 Cost packed decimal

4 20763.25 Precision: 7Signed: No Sign Pos: Trailing Separate Style: Microfocus COBOL Scale: 2 Decimals: 2

8 FileNumber text 10 file 7668 (None)

9 SerialNumber text 9 DB9785500 (None)

1-26


4 Click the Parse Data icon in the tool bar to display the Enter Connection Information window.

5 Click the down arrow to the right of the Source File/URI box.

6 Select the file you want to parse and click Open.

7 Click the down arrow next to the length box in the Parser window to open the Hex Browser window. In this window, you can scroll through the information in the file one byte at a time to determine the starting offset and record length.

Sometimes there will be data in the file prior to the first record that you will need to tell the Structured Schema Designer to ignore. To do this you will have to find where the first field of the first record begins.

Often a record will start with a recognizable numeric or text field so you should be able to find it by scrolling along to its first appearance. Once you have determined what the first field looks like you should be able to find the record length from the last byte before the next occurrence of this field.

For example, if you know each record starts with a unique numeric ID, you can determine from the following:

12345678901234567890123456789012345678901234567890123../asrf001SMITH J56.BÓ3..B + .#s.002THOMPSON f45

The first record probably begins with the 8th character and ends with the 37th. This makes the starting record offset 8 and the record length 30.

1-27


8 The hex numbers above the record and the appearance of the data are also useful in determining where the first record begins and ends. Text and a few other data types will be clearly legible. Other data types will appear as dots or odd symbols but the hex numbers above each byte can indicate what sort of data is this translates into. For details, see “Hex Values” on page 3-11 in the Structured Schema Designer User’s Guide.

9 Click the radio button in the upper left labeled Set Start of Record 1. There will be a magenta highlighted point on the ruler in the middle of the window. Click the right arrow in the horizontal scroll bar. Hold it until the magenta highlighted point moves to what you think is the first byte of the first record. That will indicate where your first record begins.

10 Make a note of the number that now appears in the Set Start of Record 1 box. That is the Starting Offset.

11 Click the radio button labeled Set End of Record 1. Hold down the right arrow on the horizontal scroll bar until the magenta highlighted point moves to what you think is the last byte in the first record. That will set the record length by indicating where the first record ends. Click OK.

12 If there was a record start offset, click the Connect to Data icon to open the Enter Connection Information window again. Leave all the other information as it is, but change the StartOffset value on the right of the window to whatever you determined in step 8, click Apply and then click OK.

13 The records should now be displayed in correct alignment. If not, you will need to go back and determine the start offset and record length again.

Troubleshooting Notes

If the Parser arrows do not exactly mark where a field’s data actually begins, this is not necessarily a problem. Look at the field’s Size and check the Contents box to make sure the field is parsed the way you want it.

If there are extra characters or if characters are truncated at the start of records, then you know you have set the Start Offset incorrectly. Extra characters mean you need to increase the Start Offset, and truncation means you need to make it shorter.

1-28


If patterns in the data slant to the left or right, you have set the record length incorrectly. If the patterns slant to the left, then you need to decrease the record length. If they slant to the right, you will need to increase it.

Summary In this Tutorial, you:

learned how to navigate the Data Parser, and

how to use the Data Parser to create a structure for unstructured data.

1-29


Map Designer Tutorial 5 - Single-record Type File to a Multi-record Type XML File

Tutorial 5 demonstrates how to create, save, and run a data Transformation Map where the Source is a single-record type and the Target is a multi-record type. Within these State records will be records for each City in these States, and within each of these will be the customers who live in those cities.

Instead of a simple one-to-one transformation as in previous Tutorials, this example involves creating records for each State contained in the data. This process involves sorting the ASCII Delimited Source file, and using Event Handlers to determine when to create each type of record in the XML Target file.

Goals You will:

sort data into logical groups,

configure a Target with more than one record type,

set and configure complex event handlers,

create a hierarchical data structure, and

become familiar with many terms used throughout the documentation.

Requirement Transform a single-record type ASCII Delimited source file to a multi-record type XML target file.

Procedure 1 From the Repository Explorer, click the down arrow beside the New icon in the toolbar. Then select Map to open the Map Designer window.



3 Click the Factory Connections tab and select ASCII (Delimited) from the list. Click OK.

4 Click the down arrow next to the Source File/URI box to open the standard Windows navigation dialog box.

1-30


5 Select Tutor1.asc and then click Open.

6 Click the Source Data Browser icon in the toolbar.

The data from the Source file will display in a rectangular grid in the Source Browser. Individual records are shown as separate rows, while individual fields are shown as separate columns.

You will see that the Tutor1.asc file contains field names in the first row of data. This is because the first record is a file header. You will need to set the appropriate Source Properties to take this into account.

7 Close the Browser Window by clicking the X button in the upper right.

8 Set the Header Properties, on the right side of the Source Connection tab, by double-clicking its current value. This will change it to True, to show that it does have a header record. Click Apply below the list of properties.

You want to create hierarchical XML records based on State, City and Customer Information, so you will need to sort the records before doing anything else with them.

9 Click the Source Keys and Sorting icon in the tool bar to open the Source Sort Expressions and Keys window. This is where you specify the key fields of the Source data file for sorting purposes. When the transformation is run, the Source records will be sorted before they are transformed.

10 Click in the first Key Expression box and click the down arrow that is displayed.

11 Select Fields ("State") from the drop down list. The data type and length will automatically be filled in. The default sort direction is "Ascending".

Map Designer defaults to using the first 16 characters in a sort field. If a field you select has more than 16 characters and you want to use more than the first 16 in a sort, you can change the text length by overtyping this number. For example, you might have records for the "American Federation of Musicians" and the "American Federation of Teachers". Because the first 16 letters in each case are "American Federat" a sort would consider these fields to be the same. You would therefore need to increase the Text Length to 30 characters.

1-31


Note The higher the number of characters used for sorting, the slower the conversion will run.

12 Click in the second Key Expression box and click the down arrow in the second Key Expression box.

13 Select Fields ("City") from the drop down list. The remaining fields for this Key Expression will automatically be filled in. The default sort direction is "Ascending". Data Type will become Text. Length will be 16.

14 You have now told Map Designer to sort the Source file first by State and then by City. Click OK.

15 Click the Target tab. Click the down arrow next to the Target Connection box to open the Select Connection Window.


16 Click the Factory Connections tab. Select XML from the list and click OK.

17 In the Target File/URI box, click the down arrow to display the standard Windows navigation dialog box. Enter the name you want this file to have. For this tutorial, it is suggested you save the Target file as Tutor1.xml.

18 Click the Open button.


You now need to set up the three record types you are going to use. These are State, City and CustomerInfo. The State record type will be the highest in the hierarchy as it contains the City record type and the City record type contains the CustomerInfo record types.

20 In the Target tree (bottom left grid), click Record Types. Now look at the bottom right grid. Under Record Name, highlight R1. Overtype R1 with State.

1-32


21 Click in the box below State. Do not click straight onto the Target tree or the information will not be saved. You will see that R1 has changed to State in the Target tree. R1 Event Handlers will also have changed to State Event Handlers.

22 Click State Fields under the State record type on the Target tree. The Target Record Layout for the State record type is displayed in the bottom right grid.

You now need to define the State record. The State records will contain the state names and city records. In the first cell under the Target Field Name column, type StateName.

23 In the upper right Source grid, Click and hold down the mouse button on the gray 7 to the left of the State source field name.

24 Drag the cursor down to the first cell in the Target Field Expression column of the lower right Target grid. Release the mouse button.

Note that when you use the drag-and-drop method to copy field information from source to target, the cursor changes to a . This also occurs when it is over a cell where the details cannot be dropped. The cursor changes to a when over a cell in which the details can be dropped.

Also note that if you have already entered a Target field name, you should drop the source information in the Target Field Expression cell rather than the Target Field Name cell as you did in earlier tutorials. This keeps the existing field name.

The field information will be copied from the source record and the expression " = Records(R1).Fields("State")" will appear in the Target Field Expression cell. The Type defaults to Character, and the Size defaults to 16.

1-33


Because the state name is going to be an attribute in the XML file, you need to change the data type. Click in the first cell under the Type column in the Target Record Types, click the down arrow that appears, then select Attribute from the drop down list that appears.

Tip To jump to Attribute in the drop down list (once it is expanded), press the A key on your keyboard. Map Designer automatically goes to the first entry starting with A in the list, you can then scroll down to Attribute.

25 Click in the first empty cell in the Target Field Name column (beneath StateName) and type City.

26 You now want to define the City record. This will contain the name and zip code of the city and the records for each customer in this city. Leaving the Target Field Expression blank, click in the second cell under the Type column, click the down arrow that appears and then select Record from the drop down list that appears.

Tip To jump to Record in the drop down list (once it is expanded), press the R key on your keyboard. Map Designer will automatically go to the first entry starting with R in the list, which is Record.

27 The following message is displayed: "Do you want to add a record type with this name?" Click Yes. A new record type called City will appear below State in the Target tree. Note that the City is indented beneath State. This shows that City records will be contained in their parent State record.

28 Click City Fields under City in the Target tree.

29 The Target Record Type for the City record type will appear in the bottom right Target grid.

30 In the first cell under the Target Field Name column, rename Field1 to CityName.

31 In the upper right Source grid, click and hold the mouse button down in the gray cell to the left of the City field.

1-34


32 Drag the cursor down to the first cell in the Target Field Expression column of the lower right Target grid. Release the mouse button.

33 The Target Field Expression cell now contains the expression "= Records(R1).Fields("City")".

34 Click in the first cell under the Type column, click the down arrow that appears and select Attribute from the drop down list.

35 Click in the first blank cell under the Target Field Name column and type Zip.

36 In the Source grid, click and hold the mouse button down in the gray cell to the left of the Zip field.

37 Drag the cursor down to the second cell in the Target Field Expression column of the Target grid. Release the mouse button.

38 The expression "= Records(R1).Fields("Zip")" will appear in the cell. The Type defaults to Character, and the Size defaults to 16.

39 Click in the second cell under the Type column, click the down arrow that appears and select Attribute from the drop down list.

40 Click in the third cell under Target Field Name and type CustomerInfo.

41 Leaving the Target Field Expression blank, click in the third cell under the Type column, click the down arrow that appears and select Record from the drop down list that appears.

42 The following message will appear: "Do you want to add a record type with this name?" Click Yes. A new record type called CustomerInfo will appear indented below City in the Target tree.

43 Click CustomerInfo Fields under CustomerInfo on the Target tree. The Target Record Type for the CustomerInfo record will appear in the Target grid.

Tip Since you will be dragging several lines of information down to populate this layout, you should delete the default information displayed when you open the CustomerInfo record type. To do this, click the green cell to the left of the row to be deleted. The row is highlighted. Press the <DELETE> key. Click Yes to confirm deleting the entry.

1-35


44 While holding down the <CTRL> key, click the blue column to the left of lines 1, 2, 3, 4, 5, 9, and 10 in the Source grid. Continue holding down the mouse button after you click on line 10. Each line will be highlighted as you click on it.

45 Keeping the mouse button held down, drag the cursor to the first cell in the Target Field Name column. It should turn into a black oblong when you are over this cell. Release the mouse button.

Tip If you have difficulty dragging multiple cells, you can drag each field individually. Or you may right-click after highlighting your Source Fields and select Copy. Then, in the first cell under Target Field Name, right-click again and select Paste.

46 This will populate the fields of the Target grid with the values from the Source grid for the customer’s Account Number, First Name, Last Name, Company, Address, Payment and Balance.

47 Click the OK button on the Invalid Field Names dialog box if you receive an Invalid Field Name error.

48 Some Source Field Names have spaces between names and XML does not allow spaces. Map Designer will offer to fix the problem. After you click OK, all spaces will be changed to underscores ("_") to create valid Target field names. The Target Field Expression column will be filled with the Source field names in appropriate notation for mapping expressions.

49 You now need to tell Map Designer to create State records only when the State changes. As you told Map Designer to sort records by State, this means you will only create a new record for each state. For example:

Table 1-1 Tutorial 5 - State Record Creation Table

Source Record

State Action required

1 AK state has changed so create AK record

2 AK state is the same so do not create record

3 CA state has changed so create CA record

4 FL state has changed so create FL record

1-36


To do this, you will use event handlers that are triggered by a change in data.

50 In the Source tree, in the R1 Event Handlers node, click the small plus sign ("+") next to Data Change Events to open the Data Change Events node.

51 Click on OnDataChange1. The Source Record R1, Event: OnDataChange1, Actions grid will appear in the upper right window.

52 Click the down arrow to the right of the Data Change Monitor cell and select Records("R1").Fields("State") from the drop down list that appears.

The State field will now be monitored so that when it changes value the event you are now going to specify will occur.

53 Click the down arrow by the box to the right of the Data Change Monitor cell and select Fire first ODC event/Suppress extra ODC event on EOF.

This tells the Data Change Monitor to fire the event on the first data change but not to fire an extra event at the end of file.

You now need to tell Map Designer what to do when the State changes.

54 Click on the ellipsis button in the first cell of the Action column on the Source Record Event Actions grid. The OnDataChange1 action definition window is displayed.

55 Select MapPut Record from the drop down list that appears. The parameters grid for this action is displayed.

5 FL state is the same so do not create record

6 TX state has changed so create TX record

7 TX state is the same so do not create record




Table 1-1 Tutorial 5 - State Record Creation Table

1-37


56 Click on the down arrow at the end of the Record Layout box and select State. The Count and Count Variable parameters are not needed for this action.

57 Click OK. The information you have set up is now displayed in the Parameters column.

Notice that the OnDataChange1 entry on the tree at the left has become bold, indicating that it contains an Event Handler to be processed. If you are looking at a transformation that someone else has created, you will be able to tell which events are used as triggers because they will be displayed in bold in the tree.

This Event Handler has now been set up to cause Map Designer to write a piece of data (in this case, the State name) to your Target file, using the State record layout, each time the value in the State field changes.

You now need to do the same thing for City data: set up an event that fires when the city name changes and set up the event action to write a record using the City record layout.

58 In the Source tree, click on OnDataChange2. The Source Record: R1, Event: OnDataChange2, Actions grid will appear in the upper right window.

59 Click on the down arrow to the right of the Data Change Monitor cell and select Records("R1").Fields("City") from the drop down list that appears.

The City field will now be monitored so that when it changes value the event you are now going to specify will occur.

60 Click on the down arrow by the box to the right of the Data Change Monitor cell and select Fire first ODC event/Suppress extra ODC event on EOF.

This tells the Data Change Monitor to fire the event on the first data change but not to fire an extra event at the end of file.

You now need to tell Map Designer what to do when the City changes.

61 Click on the ellipsis button in the first cell of the Action column on the Source Record Event Actions grid. The OnDataChange2 action definition window is displayed.

1-38


62 Select MapPut Record from the drop down list that appears. The parameters grid for this action is displayed.

63 Click on the down arrow at the end of the Record Layout box and select City. The Count and Count Variable parameters are not needed for this action.


This Event Handler has now been set up to cause Map Designer to write a piece of data (in this case, the City name) to your Target file, using the City record layout, each time the data of the City field changes.

Since you want a record for each customer, you need to make sure that Map Designer writes each Customer record it encounters. In this case you could set up another OnDataChange event triggered by the account number, as this is unique for each customer. However, in some cases you may not be certain that a field is always unique so it is safer when you want to cause every record in the source to be processed to use the AfterEveryRecord event handler.

65 In the Source tree, click on AfterEveryRecord. The Source Record: R1, Event: AfterEveryRecord, Actions grid will appear in the upper right window.

66 Click the first cell under the Action column and then click on the ellipsis button that appears. The Event: AfterEveryRecord actions definition window will appear.

67 Click on the down arrow next to Action Name and select MapPutRecord from the drop down list that appears. The parameters grid for this action is displayed.

68 Click on the down arrow at the end of the Record Layout box and select CustomerInfo. The Count and Count Variable parameters are not needed for this action.


This Event Handler causes Map Designer to write a piece of data (in this case, the CustomerInfo name) to your Target file using the CustomerInfo record for every record in the Source file.

1-39


70 Click the Save Map icon in the toolbar to open the Save As dialog box. Type a name for this transformation and click Save. Your Map is saved.

Note You may save the Map at any point prior to this, since it is such a long procedure. If it is a partial transformation (i.e., Source connection information only) or an invalid transformation (i.e., no mapping), it will not run as a Map. However, it can be saved while in progress.


Map Designer will load all the information it needs to run this transformation and then execute it. When the transformation is finished, the total number of records read and the number converted will be displayed in the status bar. In this case you will see that 114 target records (2 state records, 12 city records and the original 100 customer records) have been created from the 100 source records.

72 Click on the Target Data Browser icon in the tool bar to view the file.

73 After viewing the new file, Close the Target Browser by clicking on the X on the upper right corner. Close Map Designer by clicking on the X on the upper right corner.

Congratulations, you have just created, saved, and run a multi-record Map in the Map Designer!

Summary In Tutorial 5, you learned how to:

sort data into logical groups,

configure a Target with more than one record type,

set and configure complex event handlers,

create a hierarchical data structure, and

recognize and use many of the terms you will encounter throughout the documentation.

1-40

c h a p t e r

2
Process Designer Tutorials
Step-by-Step Tutorials for Learning Process Designer

This chapter includes two tutorials to help you learn how to use Process Designer and includes the following sections:

“Introduction to Process Designer Tutorials” on page 2-2

“Process Designer Tutorial 1 - The Basics” on page 2-3

“Process Designer Tutorial 2 - Loops and Queues” on page 2-11

2-1


Introduction to Process Designer TutorialsThese step-by-step tutorials will help you learn how to use Process Designer. The tutorials have been designed to be worked through in order, as they lead you through the basics and build on your knowledge as you work through each one.

Tutorial 1 - The Basics - introduces the graphical interface and the drag and drop method of creating processes.

Tutorial 2 - Loops and Queues - introduces the use of loops to create more complex, repeating processes. It also shows how to use a transformation, messages and queues.

Tip To get the most from the Process Designer tutorials, you should be familiar with Map Designer and have worked through Map Designer's Tutorials before using Process Designer. Because of the length of these tutorials, it recommended that you print out hard copies rather than try to follow them on screen.

2-2

Process Designer Tutorial 1 - The Basics

Process Designer Tutorial 1 - The BasicsTutorial 1 introduces the basic concepts used in Process Designer. In this tutorial, you will learn how to create a process and run it.

Goals In Tutorial 1, you will learn:

how to create a new process,

how to add a Scripting Step

how to add a Decision Step

how to add a conditional statement to the Decision Step

how to direct the output of the Decision Step to the next appropriate Step in your Process

and most important, how to run a Process.

You will use each of these basics as you build your skills in Process Designer.

Requirement Create a process that runs once each day. On Sunday, additional routines also run. For this tutorial the transformation and the Sunday routines are not actually included. Instead, comment boxes are displayed. As an additional exercise, you could create you own Transformation using the sample data.

Procedure 1 From Repository Explorer, click on the down arrow beside the New icon in the toolbar and select Process to open the Process Designer window.

A Process Canvas is displayed showing Start and Stop steps. Every process must have one start and one stop step.

Note If you start from within Process Designer instead of Repository Explorer, clicking the New icon creates a new Process without the drop-down menu step.

2 Click once on the Scripting icon on the Process Step Bar and move the cursor over the canvas. Position the cursor below the Start step and click the left mouse button. A Script step will be displayed in this position.

You can click-drag a Step to any position on the Canvas.

2-3


3 Double-click on this step to open the Scripting (properties) window.

4 Rename the icon "Run Daily Transformation".

5 Click on the Expression field and enter:

MsgBox("Run Daily Transformation")

Note If you are not familiar with the MsgBox Function, see “MsgBox Function” on page 3-184 in the RIFL Programmer’s Reference.

6 Click OK to close the Scripting window.

7 Click on the Decision icon on the Process Step Bar, position the cursor beneath the Script step and click the left mouse button again. A (conditional) Decision step will be displayed.

8 Double-click on this step to open the Decision (properties) window.

9 Rename the icon:

"Is it Sunday?"

We now need to create a condition that will be TRUE if this procedure is run on a Sunday and FALSE on any other day.

10 Click on the Expression field and enter:

DatePart("w",Now())=1

Note The parameter “w” returns “day of the week”. See the help topics for the DatePart() and Now() functions if you are not familiar with these.

11 Click OK to close the Decision window.

12 Create another Scripting step, this time to the right of the Decision step.

13 Double-click on this step to open the Scripting Properties window.

14 Rename the step:

"Sunday Only Routines".

You should now have the following diagram:

2-4


Now link the steps to show the sequence in which they must run.

15 Click on the Link icon and move the cursor over the Start step. The cursor will appear as a when it is over the canvas and when it is over a step.

16 Click on the Start step. Now move the cursor to the "Run Daily Conversion" Scripting step. Again the cursor will appear as until it is over a step, when it will appear as .

17 Click on the Scripting step. An arrow will now appear linking these steps in the order they are to be performed.

Note that the cursor is still showing as . This is so that you can link all your steps without having to repeatedly return to the Process Step Bar.

2-5


18 Create a link from Run Daily Conversion Step to Is it Sunday? Step in the same way.

19 Create a link from Is it Sunday? Step to the Sunday Only Routines function Step. In this case the following dialog box will be displayed:

20 Select True. This shows that this is the path to follow if the condition specified in the Is it Sunday? Step evaluates as TRUE.

21 Create a link from the Is it Sunday? Step to the Stop Step. Again the Decision dialog box is displayed. Select False.

22 Finally create a link from the Sunday Only Routines Step to Stop Step. Your diagram should now look similar to this:

2-6


To run the Process, click on the button. If everything has been set up correctly the following will appear.

2-7


This shows the Run Process dialog that displays details of the Process as it goes through each step. In this case the RIFL Script MsgBox shows that it has reached the "Run Daily Transformation" step. (In a real-world example a transformation would be carried out at this point and the Process would not be stopped.)

23 Click OK to continue.

The following shows the Run Process box after the process has completed on a Sunday:

2-8


The following shows the same process if run during the week:

Summary In Tutorial 1, you learned:

how to create a new process,

2-9


how to add a Scripting Step

how to add a Decision Step

how to add a conditional statement to the Decision Step

how to direct the output of the Decision Step to the next appropriate Step in your Process

and most important, how to run a Process.

You will use each of these basics as you build your skills in Process Designer.

2-10

Process Designer Tutorial 2 - Loops and Queues

Process Designer Tutorial 2 - Loops and QueuesTutorial 2 demonstrates how various components can be used together, and how to execute a Map Designer data Transformation from Process Designer.

Goals In Tutorial 2, you will learn:

how different Process Designer components can be used together,

how to execute a Map Designer data Transformation from within Process Designer,

how to use a FileFolderQueue to listen to a directory for incoming ASCII files, and

how to process data in a message and put the processed message back in a queue.

You will use these somewhat advanced skills as you gain proficiency in Process Designer.

Requirement Use a FileFolderQueue to listen to a directory for incoming ASCII files. Process the data in the message and put the processed message back in the queue as a .txt file.

To put the processed message back in the queue, the following happens:

the file is read from the folder into a message

data in the message is processed into a Target message

then the data in the Target message is written to a new file in the folder

Procedure 1 Create a directory called TUTORIAL2 in a Repository. You will see references to ../TUTORIAL2 in this section, and navigate to this directory.

2 From Repository Explorer, click on the down arrow beside the New icon in the toolbar and select Process to open the Process Designer window.

A Process Canvas is displayed showing Start and Stop steps.

2-11


Note If you start from within Process Designer instead of Repository Explorer, clicking the New icon creates a new Process without the drop-down menu step.

3 Select Process Properties from the File menu.

4 Select the Logging tab, find the Process section in the lower left of the pane, and uncheck the option labeled “Break after first error”.

Note If this option is checked, the Process will stop the first time the queue is checked and no message is found.

5 Click OK to close the Process Properties window.

6 Click-drag the Start step to the top left corner of the canvas to make room for all of the Steps we will use in this Process.

7 Right-click on the Process Variables folder and select Add. The Global Variable dialog box is displayed.

8 Create a message variable with the following details:

9 Variable Name - infile

2-12


10 Variable Type - DJMessage (In the drop-down menu.)

11 Click OK to close the box. A variable named infile was created in the Process Variables folder.

12 Right-click on the Queue Session folder and select Add. The Queue Session dialog box is displayed.

13 Create a queue that retrieves files that have the .asc extension from a directory and returns files with the .txt extension into the directory in the format outfile_datetime.txt. Enter the following details:

Leave the other options blank or accept the defaults as they are.

14 Click OK. You will now see that a queue session called MyQueue has been created in the Queue Sessions folder.

15 Click on the Queue icon in the Process Step Bar, and then click on the canvas to the right of the Start step. A Queue step will be displayed.

16 Double-click on the queue step. The Queue dialog box is displayed.

17 Enter the following details:

Table 2-1 Queue Session Details

Queue Session Details

Session name MyQueue

Type FFQueue (select from pulldown list)

Options

1. Directory ../TUTORIAL2

2. Get MsgClass asc

3. PutMsgClass txt

4. Filename Format outfile_%d_%n

Queue Details

Name GetMessage

Ignore Error? (blank)

Queue Session MyQueue (Select from the drop-down list.)

2-13


Leave the other options blank or as they are.

18 Click OK.

19 Click the Decision icon in the Process Step Bar. Next, click the canvas to the right of the Queue step. A decision step will be displayed.

20 Double-click on the Decision step. The Decision dialog box will be displayed.

21 Enter "Did we get a message?" as the name for this step.

Next, create a condition based on a property returned by another Step. In this Decision Step, you will test for a message in the queue. If a message is present, it will be processed in subsequent Steps. If no message is in the queue, the Process should loop, and check again. To test the queue for a message, use the ReturnCode property and set the Value to "0". This will only be "0" if there is a message in the queue.

22 Click on the Step Result Wizard button. The Step Decision Script Wizard dialog box is displayed.


24 Click the Add button and then OK to close the Wizard.

25 Click the OK button to close the Decision dialog box.

Action GetMessage (Select from the drop-down list.)

Parameter: Message Name infile

Step Decision Wizard Details

Step Name GetMessage (Select from the drop-down list.)

Step Property ReturnCode (Select from the drop-down list.)

Operator == (Select from the drop-down menu.)

Value 0

Queue Details

2-14


26 Click on the Function icon in the Process Step Bar, and then click on the canvas beneath the Queue step. A function step will be created.

27 Double-click on the Function step. The Function dialog box will be displayed.

28 Enter the following details.

This will cause the process to wait for five seconds (5,000 milliseconds) every time this function is encountered.

Click the OK button to close the step.

29 Click the Transformation Map icon in the Process Step Bar, then click on the canvas beneath the Decision step and lower than the Wait function. A Transformation step will be created.

30 You should now have a diagram like this:

31 Double-click on the Transformation step. The Transformation step dialog box will be displayed.

32 Enter "Transform the data" as the name for this step.

33 Click the New button to open Map Designer.

34 Using the Source Connection tab, create an ASCII (Delimited) Source connection with the Source File: djmessage:///infile. Set the Header Property to True.

35 Using the Target Connection tab, create an ASCII (Delimited) Target connection with the Target File: djmessage:///outfile. Set the Header Property to True. You will also need to create a Schema for this Target as follows:

36 Click on the Build Schema button next to the Structured Schema field. The Structured Schema Designer is displayed.

Function Step Details

Name Wait

Expression Sleep$(5000)

2-15


37 Create a schema with the following two fields:

38 Save this schema as output.ss.xml and exit Structured Schema Designer.

39 Click on the Map tab.

40 Enter the following Target Record information:

Schema Field Details

Name Type Size

Account Number Text 5

Revised Balance Text 6

Target Record

Target Field Name Target Field Expression

Account Number =Records("R1").Fields("Account No")

Revised Balance =(Records("R1").Fields("Payment")+Records("R1").Fields("Balance"))

2-16


41 Save the map as PD_tutorial2.map.xml and exit Map designer.

42 Click OK to close the Transformation dialog box.

43 Click on the Queue icon in the Process Step Bar, and then click on the canvas to the left of the Transformation step. A Queue step will be created.

44 Double-click on the Queue step. The Queue dialog box will be displayed.


Leave the other options blank or as they are.

We have now created all the steps required and need to link them as shown in the next diagram:

Queue Details

Name PutMessage

Queue Session MyQueue (Select from the dropdown list.)

Action PutMessage (Select from the dropdown list.)

Parameter: Message Name outfile

2-17


46 Save the Process as PD_Tutorial2.ip.xml.

47 Run the Process. As there is nothing in the ../TUTORIAL2 directory, the Process will loop.

48 While the Process is looping, open Windows Explorer and drag a copy of the TUTOR1.ASC file into the ../TUTORIAL2 directory.

2-18


49 The process will now complete, and the Run Process window will display the following:

Summary In Tutorial 2, you learned:

how different Process Designer components can be used together,

how to execute a Map Designer data Transformation from within Process Designer,

how to use a FileFolderQueue to listen to a directory for incoming ASCII files, and

how to process data in a message and put the processed message back in a queue.

2-19


2-20

c h a p t e r

3
Extract Schema Designer Tutorials
Step by Step Tutorials for Learning Extract Schema Designer

There are seven Extract Schema Designer tutorials. The self-paced tutorials are designed to be done in order, therefore, you should start with Tutorial 1.

“Introduction to Extract Schema Designer Tutorials” on page 3-2

“Tutorial 1 - The Basics” on page 3-4

“Tutorial 2 - Tagged Data and Automatic Features” on page 3-11

“Tutorial 3 - Columnar Data” on page 3-20

“Tutorial 4 - Floating Tags” on page 3-29

“Tutorial 5 - Columnar Data with a Footer” on page 3-37

“Tutorial 6 - Variable Length Multi Line Data Fields” on page 3-45

“Tutorial 7 - Multiple Accept Records” on page 3-51

3-1


Introduction to Extract Schema Designer TutorialsIf this is your first to time to use Extract Schema Designer or if you are seeking to understand the newer features of the Extract Schema Designer, we have provided resources to help you learn Extract Schema Designer. These resources will guide you through basic scripts and into more difficult data transformation features. Step-by-step tutorials and multiple sample scripts demonstrate Extract Schema Designer in action.

We hope that these resources will point out features that make difficult jobs easy to perform in order to reduce the amount of time you spend learning the product before attempting difficult data transformation jobs.

Be sure to take a look at the Knowledge Base and FAQ on the support page of our website.

User Driven Tutorials

Sample Scripts

You have been provided with a library of sample scripts that demonstrate how various commonly used procedures are executed in the Extract Schema Designer. These samples, since they are actual Extract Schema Designer scripts, can be used as templates for you to design similar extracts of your own.

These samples are available for your study via either of two methods:

1 They are on the installation CD. They are not part of the normal Extract Schema Designer install routine, however, so you must install them separately in order to use them.

2 They are downloadable from our website. To install, double-click on the self-extracting .exe file after a successful download.

After the samples have been installed on your system, choose Change Database from the File menu in the Extract Schema Designer, and select (the Extract Schema Designer installation directory)\SampleData\camp80samples.mdb. This will make all of the sample scripts available to you in the script manager of the Content Designer. Any time you want, you can return to the master database for the Extract Schema Designer by choosing Change Database from the file menu and selecting (the Extract Schema Designer installation directory)\ConvSpecs\Extractor80.mdb.

3-2

Introduction to Extract Schema Designer Tutorials

Go to our website for the latest product news, downloads, and maintenance releases. See Website Addresses.

3-3


Tutorial 1 - The BasicsTutorial 1 is designed to guide the new user through the basic steps necessary to create and save a script file in the Extract Schema Designer. Later tutorials will be more detailed.

This tutorial presents the fundamental concepts and procedures necessary for understanding and using the Extract Schema Designer. It is HIGHLY recommended that you do this tutorial first. The sample file is a tagged list but the basic procedures will be useful regardless of what sort of report you are dealing with.

Procedure

1 From Map Designer, select Extract Schema from the New Object pull down list or from the Stand-alone Extract Schema Designer, select File > New Extract. The Select the Text File window will display.

2 At the Select the Text File window, click on the down arrow next to the data box and navigate to TUTOR1.REP in the default installation directory (…\Common800 directory). Click OK.

This will open the report in the Extract Schema Designer Data Panel. The Source Options window will open as well.

3 While in the Source Options window, click on the Display Choices tab. At the display tab, turn Pad Lines ON by clicking on it once to display a check mark in the box. Click OK.

4 Select Preferences from the toolbar, to verify Close Definition Dialogs on Add/Update is ON. Now you are ready to begin.

5 The first step in defining many extract scripts is to determine the line of data that marks the end of a record. In this case, the line with the word "Category" in it is the last line of the first record.

6 After you identify the end of the record, Define the Line Style for that line by marking the information that makes that line unique. In every record in this data, the last line contains the word "category".

a. Highlight a recognition pattern. – Highlight the word "Category:" by dragging the mouse over it (and the colon following it) with the button depressed and releasing it at the end.

3-4

Tutorial 1 - The Basics

b. Select New Line Style. – Right click anywhere in the Data Panel, the large white area of the screen) to open a menu.

To open the Line Style Definition window, select Define Line Style from the pop up menu and click New Line Style from the second pop up menu.

Notice Extract Schema Designer has already formed line recognition rules based on the information you highlighted. It will look for all lines that contain the word "Category:" in columns 15 through 23.

c. Specify Line Action. – Now you must indicate that the Extract Schema Designer should Accept the record at this point, ending one record and beginning the next.

Click on the Line Action tab.

Click on the Accept Record radio button. Since the category line does contain information that you will want included in the record above, leave the setting at including this line’s fields.

d. Click Add. – Click on the Add button. The Line Style Definition window will close.

Notice that the Line Style name, Category, now displays in the Line Style Column (the yellow column on the left of your screen) to mark that line as matching the "Category:" Line Style pattern. A bold green arrow is displayed designating that this is the ACCEPT Record line. If you scroll down in the data, you will notice that each line that matches the pattern you defined has been automatically marked with the Category Line Style.

7 Next, let’s define another line that contains useful data. The WINTECH line does not contain information that is useful to us and can be ignored.

In the TUTOR1 file, the first line of text that contains pertinent data is the line with the report date "13-Jul-95". The dashes (hyphens) in this line make it unique and remain consistent even if the date changes in later records.

a. Highlight a recognition pattern. – Highlight the first dash by dragging the mouse over it.

b. Select New Line Style. – Right click in the Data Panel (the large white area on the right).

3-5



The Line Style Definition window will open, notice that a pattern has been created based on what you highlighted. The Extract Schema Designer will look for any line that contains a dash in column 13.

c. Specify Line Action. – Click on the Line Action tab; leave the option set to Collect Fields. The Collect Fields option will cause any fields defined on this line to be included in the final grid. Collect Fields is the action you will want for the majority of lines in this report.

d. Click Add. – Click the Add button. Notice that the name Style1 and (gray barbell) will appear in the Line Style Column (the yellow column on the left). The name and icon help you identify lines of text that have been defined and to indicate at a glance the Line Action for that line.

8 Now, you need to define the rest of the important lines. The majority of the lines in this report have tags at the beginning that indicate the type of information that line contains, and make it easy to define a pattern for that line.

Locate the line of text that contains the text "Problem No:". This is a handy tag you can use to identify this line.

a. Highlight a recognition pattern. – Highlight the text "Problem No:" with the mouse.

b. Select New Line Style. – Right-click in the Data Panel (white right side of the Extract Schema Designer window).


Notice that the Extract Schema Designer has generated a Line Style recognition pattern based on the highlighted text "Problem No:". The Extract Schema Designer has also used the first words "Problem_No" to name the Line Style. You may rename the Line Style if you wish, but Problem_No will do.

3-6


a. Specify Line Action. – The Extract Schema Designer automatically selects Collect Fields as the line action. Since this is the option you want on most of the lines in this report, accept the default.

b. Click Add. – Click Add. The Line Style name and (gray barbell) will appear.

9 Repeat the steps in #7 for each of the lines in the first record. Remember, the "Category" Line Style has already been defined, and it is the ACCEPT Record line.

a. Highlight a recognition pattern.

b. Select New Line Style.

c. Specify Line Action.

d. Click Add.

When you have defined Line Styles for 14 lines of the first record, you are ready to define the Data Fields.

10 Now that you have given the Extract Schema Designer the pattern information it needs to identify the lines in the report, you need to define what part of that line you consider useful.

a. Highlight the data. – Locate the line containing the date of the report, the line shows only the report date so all of the text on that line is important.

Highlight the entire date by dragging the mouse over it. The column numbers will show at the bottom right part of the screen. Columns 10 through 19 contain the date.

b. Select New Data Field. – Right-click in the Data Panel.

To open the Field Definition window, select Define Data Field from the pop up menu and click New Data Field from the second pop up menu

Notice the Field Definition option is set to Fixed Column in both the Start Rule and End Rule tabs. The Data Field starts in column 10 and ends in column 19 exactly where you highlighted.

c. Name the Field. – The Field Name defaults to "Style1_1" indicating that this is the first field on the Style1 line. Change the default name to a more descriptive name, ReportDate by typing it in the box labeled Field Name.

3-7


d. Click Add. – Click the Add button. The Field Definition window will close. Notice that the barbell to the left of the line will change color to red and the field area you defined will now be red. The new color help you identify lines that have been defined and to indicate at a glance the lines location.

11 Now, you need to define the remaining Data Fields.

a. Highlight the data. – On the "Problem_No" line, highlight from column 25 to 30. This will grab enough space to include any larger numbers that might occur in later records.

b. Select New Data Field. – Right click in the Data Panel. Select Define Data Field à New Data Field.

Notice the Field Definition defaults to Fixed Column in both the Start Rule and End Rule tabs, starting in column 25 and ending in column 30.

c. Name the Field. – The default Field Name is "Problem_No_1". Problem_No is a descriptive name, but there is only one field on this line so the number, 1, is unnecessary. Click in the Field Name box and backspace twice to delete the number and underscore.

d. Click Add. – Click the Add button. The Field Definition Window will close. The barbell to the left of the line

and the field will change color.

12 Repeat the steps in #11 above for each line of text in TUTOR1.REP that contains the tagged data.

a. Highlight the data.

b. Select New Data Field.

c. Name the Field.

d. Click Add.

Field Name Start Rule End Rule

TECHIE 25 53

STATUS 25 53

MMDDYY 25 32

TIME 25 32

SERIAL_NO 25 53

VERSION 25 39

3-8


CUST_NAME 25 53

COMP_NAME 25 53

PHONE_NO 25 53

SOURCE_TYPE 25 53

TARGET_TYPE 25 53

CATEGORY 25 53

You have now defined all the Data Fields in the TUTOR1.REP report file.

Note that the icon next to the Category line will change to . This indicates that this line is the Accept record and has a data field defined, but the data field has not been officially assigned to that Accept record.

13 Click on the Browse Data Record button in the toolbar. Since there is only one Accept record, a message window will appear saying, "Fields Assigned to Accept Record Category." Click OK. All of the Data Fields will appear in the Data Browser window for you to preview your data in a normal database format and to verify you have defined everything correctly. Close the Data Browser window by clicking the X in the upper right corner.

Once you have assigned your data fields to an Accept record, the icons next to your Collect lines will change to , and the icon next to your Accept line will change to . The blue in the barbells indicates that the data field has been assigned to an Accept record.

14 The fields in the Data Browser window are not in the order you wish them to appear. To change the order, select Field in the toolbar and click Export Field Layout from the drop down list that appears. To move the fields to their desired positions, press and hold the left mouse button on the Field Name section of the "Category" entry. While still holding down the left mouse button drag "Category" to the bottom of the list and release the mouse button. You will see a special symbol ( ) while you are dragging the field name to its proper place.

15 You may now reopen the Data Browser window (click on the icon) to view your export fields as they will appear in your database format.

3-9


16 You are now ready to save your new extract script. (New extract scripts that have not been previously saved in Extract Schema Designer will show as "Extract: Extract1" in the title bar.)

To save an extract script:

a. Click on Save Extract button in the Tool Bar. The Save Extract window will display.

Notice that the Extract File Name has defaulted to your workspace, however, the extension has been changed to ".cxl". This is consistent with standard naming for the Extract Schema Designer scripts. You can change the Extract File Name, but the extension should remain ".cxl".

b. You can accept or change the default Author name.

c. Enter a description of "Tutorial 1" (optional) and click the OK button. Your new extract script is now saved.

17 Now using Map Designer, you can use your Extract File Name to create new maps.

Congratulations, you have just created and saved your first extract script in the Extract Schema Designer!

To close the Tutorial 1 extract script, click the Close Extract button in the toolbar.

Exit the Extract Schema Designer – from the menu, select File > Exit.

OR

You may create another extract script or see the other Tutorials for instructions on building different kinds of extract scripts.

3-10

Tutorial 2 - Tagged Data and Automatic Features

Tutorial 2 - Tagged Data and Automatic FeaturesTutorial 2 is designed to guide the new user through the steps necessary to create and save a script file using the Extract Schema Designer’s automatic processes. The source file for this tutorial is the same tagged-list as “Tutorial 1 - The Basics” on page 3-4.

This tutorial introduces some of the useful timesaving features of the Extract Schema Designer that read and flattens a data file that contains tagged data. It is useful to anyone ready to learn about the Extract Schema Designer's more advanced features.

Procedure

1 Open a New Extract by clicking the New Extract button or from the menu select File > New Extract.

2 At the Select the Text File window, click on the down arrow next to the data box and navigate to TUTOR1.REP in your default installation directory (…\Common800). Tutorial 2 shows some quicker, more automatic ways to parse the same tagged-data used in “Tutorial 1 - The Basics” on page 3-4.

Click OK to display the data in the Data Panel of the Script Designer window. The Source Options window will open as well.

3 While in the Source Options window, click on the Display Choices tab. At the display tab, turn Pad Lines ON by clicking on it once to display a check mark in the box. Click OK.

4 Select Preferences from the toolbar, make sure Close Definition Dialogs on Add/Update is ON. Now you are ready to begin.

5 The first step in defining most extract scripts is to determine the line of data that marks the end of a record. A Line Style should be defined for this line of text, and the Line Action ACCEPT Record [Including] This Line’s Fields should be selected. In the TUTOR1 data file, the line of text that contains "Category" marks the end of each record.

3-11


a. Highlight the pattern and the data. – Highlight the word "Category:" by dragging the mouse over it (and the colon following it) with the button depressed and releasing it at the end. That should highlight everything from column 15 to about column 45. Check the indicator in the lower right corner of the screen for column locations.

b. Select Parse Tagged Data. – Right click anywhere in the Data Panel, the large white area of the screen) to open a menu.

c. Select Define Data Field from the pop-up menu and from the second level pop-up menu select Parse Tagged Data.

The Extract Schema Designer will automatically define a Line Style with the word "Category" and a colon in columns 15 through 23 as the recognition pattern, and a Line Action of Collect Fields and name it Category. It will also create a Data Field that will collect any data on that line beginning at column 24, one space after the colon, and going to column 45, and name it Category_1. You can see where the field is now defined because it will turn red on your screen.

Obviously, this saves you several steps, but it is all done without you seeing it, which makes it difficult to check for errors. If you wish to check the Data Field definition, you can double click on the field itself, the red part, and the Field Definition window will open. Make any necessary changes, then click Update, or simply click Close if everything is acceptable.

If you scroll down in the data, you’ll notice that each line with the "Category" Line Style also has a "Category" Data Field that starts in column 25 and extends for 20 positions. The first Data Field in any line will be red. For more information on color coding of defined Data Fields, see “Suggested Approach - Defining Data Fields” on page 2-32 in Extract Schema Designer User’s Guide.

There will also be an icon to the left of the Line Style name. This icon indicates that this Collect line style has data fields defined, but the data fields have not been assigned to an Accept Record.

3-12


6 Since the Category Line Style is the last line of the record, the Line Action should be Accept Record. When the Extract Schema Designer creates a line style automatically, it makes the line style Collect Fields, so it needs to be changed.

a. Open the Line Style Definition window. – Double click on the Line Style Name, Category in this case, in the Line Style Column, the yellow column on the left part of your screen. The Line Style Definition window will open.

b. Adjust Settings. – Click on the Line Action tab. Select ACCEPT Record [Including] This Line’s Fields from the list of choices.

c. Click Update. – Now click on the Update button.

Notice the icon displayed to the left of the "Category" Line Style Name in the Line Style Column has changed. This icon indicates an ACCEPT Record Line Style that contains defined Data Fields, but the Data Fields have not been assigned to an Accept Record.

7 Since an ACCEPT Record Line Style has been defined, all lines that it matches that have defined Data Fields can now be viewed in the Browse Data Record. Click the Browse Data Record Button on the Tool Bar. Since there is only one Accept Record, a message window will appear saying, "Fields Assigned to Accept Record Category." Click OK. The browser will display the value of one Data Field "Category" (the only Data Field defined so far) for each of the 10 records in the file. Click on the X in the upper right corner to close the browser window.

8 Now it’s time to define the rest of the data in the record. Let’s start with the header information. This time, assume that the first line contains information you want.

a. Highlight the data. – Highlight the report name WINTECH on line 8 by highlighting positions 11 through 17 with the mouse.

b. Select New Data Field. – Right-click in the Data panel, select Define Data Field from the pop-up menu and from second level pop-up menu, select New Data Field. The Field Definition window will open.

3-13


c. Name the Field. – The default Data Field Name is highlighted. Since there is no tag on this line, the Extract Schema Designer used the data itself as the Line Style name and Data Field name. Change the field name to ReportName by typing it in Data Field Name box.

d. Click Add. – Click the Add button.

9 To define the report date Data Field, follow the same steps as above except highlight from columns 11 to 19 and name the field ReportDate.



c. Name the Field.

d. Click Add.

10 The automatically generated Jul95 Line Style needs to be updated to make it more generic for different report dates.

a. Open the Line Style Definition window. – To edit the "Jul95" Line Style, double-click on Jul95 in the Line Style Column. The Line Style Definition window will open. (Notice that the Pattern for this Line Style looks for "13-Jul-95" in columns 11 to 19.)

The cells in the grid can be sized to view the information better. To size a column, position the mouse over the line in the gray border of the grid where the column headings are. The mouse pointer will become a bold vertical bar with arrows pointing to the left and right. Hold down the mouse button and drag the edge of the column to the left or right. Release the mouse button when the column is the desired size. You can adjust the height the same way using the gray border to the left where the triangle and asterisk are.

b. Adjust Settings. – To change the pattern to look for a line with any date with the "dd-mmm-yy" format, click once in the Look For? cell on the first row of the grid ("13-Jul-95" is currently displayed there). A down arrow will appear on the right side of that cell. Click on that arrow and the Pattern Builder window will open.

3-14


The Type column defaults to literal. Leave it alone. <TAB> to the Value cell. Type a dash - (hyphen). Change the values of both the Begin and End cells to 13 by tabbing to them and typing in the correct number. Click OK. (Notice that the Look For?, Begin, and End values have changed in the Line Style Definition window to reflect the changes made in the Pattern Builder window.)

Add a new row to the Line Style Definition grid by clicking in the And/Or cell in the second row. Accept the value default of And.

Click in the Search What? cell of the second row and click the down arrow that appears. Select Column Range (m-n) from the displayed list.

In a similar fashion select Contains from the list displayed in the Operator cell of the second row.

Click on the arrow in the Look For? cell of the second row to display the Pattern Builder window again. <TAB> to the Value cell and type in a dash (-) and change the Begin and End values to 17. Be careful to only enter a dash in the Value cell and don’t leave any spaces around it. Click OK.

The line style definition should now match any line with a dash in position 13 and 17.

c. Click Update. – Click the Update button to save the changes to the ReportDate Line Style.

11 In step 3 above, you defined the "Category" Data Field by selecting the Category Field Tag, the Tag Separator, and the data, and selected Parse Tagged Data in the second level pop-up menu. (This is usually the easiest way to define a Line Style and Data Field on a line of text that includes tagged data.) To show a different approach select the entire "Problem_No" line and parse as tagged data.

a. Select the line. – Select the entire Problem No line by left clicking on that line in the Line Style Column (the left yellow column).

b. Select Parse Tagged Data. – Right-click in the Data Panel (the large white part on the right) and select Define Data Field - Parse Tagged Data.

3-15


The Line Style pattern the Extract Schema Designer automatically creates will look for "Problem No:" in positions 13 through 23. You can double click on Problem_No if you want to check it. Then click the Close button to close the Line Style Definition window.

c. Open the Data Field Definition window. – To display the Field Definition window to view the information for the "Problem_No" Data Field that was automatically generated, double-click anywhere in the Data Field where the letters are red. Notice that the end rule is 52. This is larger than the "Problem No" Data Field needs to be, because it is defining the size of the Data Field all the way to the right margin of the report.

d. Adjust Settings. – Change the end rule of the "Problem_No" field to 30.

e. Click Update. – Click the Update button. (Notice the selected area on the Data panel for the "Problem_No" Data Field is much smaller after the update.)

12 Now, you can go ahead and define Data Fields and Line Styles for the Techie, Status, MM/DD/YY, Time, Ser #, Version, Customer Name, Company Name, Phone #, Source Type, and Target Type Tagged Data Fields. You can use either the steps in #11 or in # 3.

a. Highlight the pattern and the data. – Highlight the Field Tag, the Tag Separator, and the data by dragging the mouse with the left mouse button depressed from the beginning of the Tag to the end of the Data Field (remember to extend out to the right to catch wider data in subsequent records).

b. Select Parse Tagged Data. – Right-click in the Data Panel and select Define Data Field from the first level pop-up menu. Select Parse Tagged Data from the second level pop-up menu. (The Extract Schema Designer will create a Line Style Definition and a Data Field Definition for you.)

or

a. Select the line.

b. Select Parse Tagged Data.

c. Open the Field Definition window.

d. Adjust Settings.

e. Click Update.

3-16


Things to remember when defining Data Fields and Line Styles in tagged data:

When the Extract Schema Designer automatically creates Data Fields it uses the positions you have highlighted to determine the length of the Data Field. Be sure and leave enough space for data in subsequent records that are wider than the text you are currently selecting. For example, the Techie Name in the first record is John. In a subsequent record it could be Alexander Graham Bell IV.

For tagged data, everything in the selection to the left of the Tag Separator is the Field Tag and everything to the right of the Tag Separator is the Data Field.

When a Line Style is created, it is not just for the line you are working on but also for any line that matches the Line Style that is generated. This means that when you create a Line Style that looks for "Techie:" in columns 17 to 24, and there is a Data Field defined for that Line Style in columns 26 to 55, all lines that have "Techie:" in columns 17 to 24 will have a Data Field in columns 26 to 55.

Notice that the Extract Schema Designer named the MM/DD/YY, Ser #, and Phone # Data Fields and corresponding Line Styles MMDDYY, Ser, and Phone. Also, Data Fields with embedded spaces were named with the spaces removed. This was done because Field Names can only contain letters, digits and underscores. Also scroll down in the Data panel and see how the rest of the data is being defined.

13 Use the Browse Data Record to view all of the defined Data Fields. Since there is only one Accept Record, a message window will appear saying, "Fields Assigned to Accept Record Category." Click OK. Now all of the Data Fields that you defined in the report appear in the browser. Notice that since you defined the "Category" Data Field first it appears first in the output data. Close the browser by clicking on the X in the upper right corner of the window.

14 To move the "Category" Data Field to the end of the record (and any other fields that are not in their desired position), select File > Export Field Layout option from the Menu Bar. When this option is selected, a window will open that shows the Data Field names and their position. To move the "Category" Data Field, press and hold the left mouse button on the Field Name section

3-17


of the "Category" entry. While still holding down the left mouse button drag "Category" to the bottom of the list and release the mouse button. You will see a special symbol while you are dragging: . The "Category" Data Field moves from position 1 to position 15. Click on the OK button to close the Export Field Layout window.

15 Open the Browse Data Record again. Notice that the Category Data Field is now the last field in the record. You can use the horizontal scroll bar to view the Data Fields to the right. Close the Browse Data Record window.

16 You are now ready to save your new extract script. Click the Save Extract button in the toolbar. The Save Extract window will display.

Navigate to your default installation directory (…\Common800). Name the extract script file TUTOR2. Notice that the Extract Schema Designer suffix is .cxl.

The Author name is a default, but can be changed.

Enter a description of "Tutorial 2" if you wish and click the OK button. Your new extract script is now saved.

Congratulations, you have just created and saved an extract script in the Extract Schema Designer!

To close the Tutorial 2 script, click the Close Script button in the toolbar.

You may now exit the Extract Schema Designer by selecting File > Exit from the menu.

OR

3-18


You may create another extract script. See the other Tutorials for instructions on building different kinds of scripts.

Tip This file can be parsed even more automatically. If you wish to try it, push the Clear Line Styles button. Try highlighting all the tagged data in the entire first record, beginning with the Problem No line and highlighting all the way down to the Category line. Be sure to catch all the field tags and data plus some extra space all at once. Right click and select Define Data Field/ Parse Tagged Data. The Extract Schema will create several new line styles and data fields at once. This method only works in cases of highly structured and consistent data. It can be a great time saver when conditions are ideal.

3-19


Tutorial 3 - Columnar DataTutorial 3 is designed to guide the new user through the steps necessary to create and save a script file in the Extract Schema Designer that reads and flattens a report that contains columnar data. In Tutorial 3, you will convert the data in a columnar report file, to a flattened format, using the more automatic features of the Extract Schema Designer.

This tutorial introduces more of the timesaving features of the Extract Schema Designer. Since a great many report formats contain columnar data of some kind, it is highly useful to anyone who wants to use the Extract Schema Designer.

By following the steps outlined below, you will become familiar with both the process of creating an extract script and the terms used throughout the documentation.

Procedure

1 From the Extract Manager window, open a New Extract by clicking on the New Extract button in the toolbar. The Select the Text File window will display.

2 At the Select the Text File window, click on the down arrow next to the data box and navigate to TUTOR3.REP in your default installation directory (…\Common800) Click OK. The report file will load and display in the Data Panel (the large white space in the Extract Designer window). The Source Options window will also open.

3 In the Source Options window, click the Display Choices tab. At the display tab, click Pad Lines once to display a check mark in the box. Click OK. Select Preferences from the toolbar to verify Close Definition Dialogs on Add/Update is ON (checked). Now you are ready to begin.

4 Unlike the previous tutorials, this file will have multiple Accept Record line styles in a single page of the report. The primary data record information is in the table detail lines. Each line is essentially a record. Each of these will be an Accept Record line. To create a Line Style and Data Fields for the detail lines:

3-20

Tutorial 3 - Columnar Data

a. Select the line. – Select the first detail line (it begins with SALES/MARKETING) by clicking in the Line Style column (the narrow yellow stripe on the left) immediately to the left of the line. This should highlight the entire line of text.

b. Select Parse Columnar Data. – Right click in the Line Style Column (the yellow part of the screen on the left) and from the pop-up menu, select Parse Columnar Data.

The Extract Schema Designer will do the following:

Divide the line into 7 Data Fields using spaces as a column separator. The Data Fields will be given default field names SALES/MARKETING_1 through SALES/MARKETING_7.

Notice in the Data Panel that the Data Fields on the line will be different colors to indicate that they have been defined. Also, each Data Field will be underlined with a dashed colored line.

Create a Line Style for the line. The Line Style that is automatically created will have a default Line Name of " SALESMARKETING". It will identify all lines in the report that have the words "SALES/MARKETING" in positions 1 through 16.

5 To view the definitions of the Data Fields created, double click on the colored sections of the line. For example, Double clicking on the green numbers "75,249" in the Data Panel will bring up the SALES/MARKETING_2 Data Field in the Field Definition window. SALES/MARKETING_2 is the default Data Field name given to the second Data Field in the SALESMARKETING line. It starts in position 20 and ends in position 27. Since it is defined for the Line Style "SALESMARKETING", only lines that match that recognition pattern will contain this Data Field in positions 20 through 27.

6 The Browse Data Record uses the Data Field names as column headings for the Data Fields, so it is a good idea to change the Data Field names for SALES/MARKETING_1 through SALES/MARKETING_7 to more descriptive field names.

To change the Data Field Names:

3-21


First, so you don’t have to open and close the Field Definition window several times, from the menu select Preferences and turn click once on Close Definition Dialogs on Add/Update to turn the option OFF.

a. Open the Field Definition window. – Double click on one of the Data Fields in the SALESMARKETING line to open the Field Definition window.

b. Name the Data Field. – In the Field Definition window highlight the default field and replace it with the corresponding descriptive name given below.

c. Push Update. – Click the Update button.

d. Select the next Field. – To select the next Data Field, click on the arrow beside the Field Name box to display a pull down list of Data Fields that have been defined for the current Line Style. Select the next Data Field.

e. Repeat steps 2, 3, and 4. – Name the next Data Field, click Update, and select the next Data Field. Continue until you have renamed all the fields.

f. Click Close. –To close the Field Definition box, click the Close button when you have updated all the fields.

Here are the suggested names for the Data Fields:

SALES/MARKETING_1 Department

SALES/MARKETING_2 Team1





SALES/MARKETING_7 DepartmentTotal

7 To view the new Line Style SALESMARKETING, double click on the name SALESMARKETING in the Line Style column (the yellow column on the left of your screen). The Line Style Definition window will open.

Notice the SALESMARKETING Line Style is recognized by a pattern where columns 1 to 16 contain the text "SALES/MARKETING". The SALESMARKETING Line Style requires 3 updates. Steps 8-10will take you through these changes.

3-22


8 First, while still in the Line Style Definition window, change the Line Style Name SALESMARKETING to "Detail" by highlighting SALESMARKETING in the Line Name box and replacing it with "Detail". Click the Update button to save this change.

Notice that the Line Style name in the Line Style column, on the left of the Script Designer window changes from SALESMARKETING to Detail.

Notice in the Line Style column of the Script Designer window that a colored barbell appears beside the line. This indicates that this is currently a Collect Record line with defined, but unassigned, Data Fields.

9 Still in the Line Style Definition window, click on the Line Action tab and click the radio button ON beside the ACCEPT Record Including option. Click the Update button to reflect the change in the Script Designer window.

Notice in the Line Style column of the Script Designer window that a different icon appears beside the line. This icon is green arrow surrounding the collect barbell, and indicates an ACCEPT Record line with defined, but unassigned, Data Fields.

10 The Detail Line Style will only match lines that have SALES/MARKETING in columns 2 through 16. That is the recognition pattern that the Extract Schema Designer created automatically, but it is not the pattern that is needed in this case. The pattern needs to be general enough to match all of the detail lines in the text, but specific enough to match only the detail lines. You need to update the pattern so that the Line Style will match all of the detail lines excluding the TEAM TOTALS line. To do this follow these steps:

a. Analyze the lines. – Analyze the detail lines to find what makes them unique in comparison to other lines in the text. Things to look for are position of the Data Fields, contents of the Data Fields, anything that is consistent for each of the detail lines but not contained in non-detail lines. For example, the detail lines contain:

Commas in positions 24, 34 and 75 on every line.

Only letters, white space, and a / in columns 2 through 18.

3-23


Only digits, white space, and commas in columns 20 through 79.

A digit in position 78.

An upper case letter in each of the first 5 positions.

Of all of the above observations, creating a pattern to look for uppercase letters in the first 5 positions is the best way to go. Here are some reasons why:

Defining a pattern that would check for commas in positions 24, 34, and 75 would require 3 pattern lines and probably would not match every detail line in subsequent reports. Suppose in this same report (created a week later) Team 2 of the Development department went to a pre-paid weeklong class and they only spent 100 dollars on supplies for the class. This means that a comma would not be in position 34 of that detail line so it would not match the Line Style, and the essential data on that line would be lost.

Defining a pattern to check for letters, white space, and a / in columns 2 through 18 would require three pattern lines and would also match the column heading line.

Defining a pattern to match lines that contains at least one digit in positions 20 through 79 and do not contain letters or / would require 3 pattern lines and it would match the detail lines. However, it also matches the Team Totals line.

Defining a pattern to match lines that contain a digit in position 79 would match the detail lines and the Team Totals line.

b. To define a pattern that looks for upper case letters in positions 2 through 6 follow these steps:

(1) Open the Line Style Definition window. – The window should already be open, so click on the Line Recognition Rules tab if not in that section.

(2) Define a recognition pattern.

(3) Click Update. – Click the Update button in the Line Style Definition window to update the Detail Line Style. Notice that Detail now appears beside each of the detail lines in the Line Style column.

3-24


c. Click Close to close the Line Style Definition window.

11 To view the output Data Fields that have been defined so far, click on the Browse Data Record button in the Tool Bar. Since there is only one Accept Record, a message window will appear saying, "Fields Assigned to Accept Record Detail." Click OK. The browser opens with the seven Data Fields defined on the Detail lines displayed in a grid. If there appears to be blank cells in the source data browser, locate the divider line to the right of each column in the header row and double-click on the divider line. The column will automatically resize to display the entire contents of the field. To close the browser, double click on the system button in the upper left corner of the window.

Note that the icons to the left of the Line Style names have now changed. Accept Line Styles now have the icon. The blue in the barbells indicate that the Data Fields have been assigned to an Accept Record.

Table 3-1 To Build a Line Style Recognition Pattern

Steps

1. Open the Pattern Builder window. – In the Line Style Definition window, click once in the Look For? cell in the first row of the grid and click the down arrow that appears. This will display the Pattern Builder window.

2. Select the type of data. – In the Pattern Builder window, click in the Type cell and click the down arrow that appears. Click the arrow to display the allowable values for the Type field and select character class from the list. This tells the Extract Schema Designer what kind of data it will need to match for that line style to be valid.

3. Enter a value. – Tab to the Value cell and click the arrow to display the allowable values for the Value field. Select upper case letters from the list. This tells the Extract Schema Designer the specific data it will need to match for that line style to be valid.

4. Enter the count and location. – Change the value in the Count cell to 5 by highlighting the value there and typing a 5. Change the value of the Begin field to 2 and the End field to 6 the same way. This tells the Extract Schema Designer where to look for the data you specified and how many of that particular data must be found for the line style to match that line.

5. Click OK. – Click the OK button in the Pattern Builder window to save the changes and return to the Line Style Definition window.

3-25


12 Now it’s time to define the rest of the data in the record. Let’s start with the report title. To define the ReportTitle Data Field:

a. Highlight the data. – Select the report title ABC CORPORATION BUDGET on line 1 by highlighting it with the mouse in the Data Panel.

b. Select New Data Field. – Right click in the Data Panel (the large white part of the screen). From the pop-up select Define Data Field and from the second-level pop-up menu, select New Data Field. The Field Definition window will open.

c. Name the field. – Change the default name to a more descriptive name, ReportTitle.

d. Click Add. – Click the Add button. The Extract Schema Designer will take the selected text and Data Field name to automatically define a Data Field named ReportTitle and a line style as well, named ABC_CORPORATION_BUDG.

e. Click Close to close the window.

f. Double click on ABC_CORPORATION_BUDG in the Line Style Column to display the Line Style Definition window.

Notice that the Extract Schema Designer automatically created a recognition pattern that looks for the literal "ABC CORPORATION BUDGET" in positions 27 through 48.

g. Type in ReportTitle and click the Update button to change the name.

h. Click Close to exit the Line Style Definition window.

13 To define the ReportDate Data Field:

i. Highlight the data. – Select the report date "10/26/95" on line 2 by highlighting the text with the mouse.

j. Select New Data Field. – Right-click in the Data Panel, select Define Data Field from the pop-up menu and choose New Data Field from the second-level pop-up menu. The Field Definition window will open.

k. Name the field. – Change the default name to ReportDate.

l. Click Add. – Click Add.

The Extract Schema Designer will take the selected text and entered Data Field name and automatically define a Data Field and a Line Style.

3-26


Double-click Style1 in the Line Style Column to display the Line Style Definition window. Notice that the Extract Schema Designer automatically created a recognition pattern that looks for the literal "/" in positions 35 and 38. Rename the Line Style to ReportDate.

Click the Update button to change the Line Style name. Click the Close button to close the Line Style Definition window.

14 Click the Browse Data Record button on the Tool Bar. Since there is only one Accept Record, a message window will appear saying, "Fields Assigned to Accept Record Detail."

15 Notice that the ReportTitle and ReportDate are the last Data Fields in each record. This is because they were defined last. To make the ReportTitle and the ReportDate Data Fields the first and second Data Fields follow these steps.

16 Close the browser by double clicking the system button in the upper left corner of the Browse Data Record window or clicking the X in the upper right corner.

17 Choose the Export Field Layout option from the Field Menu. A list of Data Field names and positions will display.

Move the ReportTitle field by pressing and holding down the left mouse button on it and dragging ReportTitle to the top of the list. When you release the mouse button ReportTitle should be repositioned in the first position. Note that the dragged Data Fields will be inserted above the Data Field you are positioned on when you release the mouse button.

Move the ReportDate Data Field to the second position by dragging it to the current Data Field in position 2 and releasing the mouse button.

Click the OK button to save these changes.

18 Check the Data Fields by clicking the Browse Data Record button in the Tool Bar. Notice the ReportTitle and ReportDate Data Fields are now at the beginning of each record. Close the Browse Data Record window by double clicking on the system button in the upper left corner of the Browse Data Record window.

19 You are now ready to save your newly defined extract. To save an extract script:

3-27


Click on the Save Extract button in the Tool Bar. The Save Extract window will display.

Notice that the Extract File Name has been defaulted for you to your workspace, however, the extension has been changed to ".cxl". This is consistent with standard naming for the Extract Schema Designer scripts. You can change the Extract File Name, but the extension should remain ".cxl".

The Author name is defaulted for you but can be changed.

Enter a description of Tutorial 3 and click the OK button. Your new extract script is now saved.

Congratulations, you have just created and saved your Tutorial 3 extract script in the Extract Schema Designer!

You can now close the Tutorial 3 script by clicking the Close Extract button in the toolbar.

From here you can exit the Extract Schema Designer by selecting File > Exit from the menu.

OR

You may create another extract script. For instructions on building different kinds of extract scripts, see the other Tutorials.

3-28

Tutorial 4 - Floating Tags

Tutorial 4 - Floating TagsTutorial 4 is designed to guide the new user through the steps necessary to create and save a script in the Extract Schema Designer that reads and flattens a data file that contains floating tag data in a variable-length ASCII report.

This tutorial would be useful to anyone likely to be working with floating tag data.


Procedure

1 Open the Extract Schema Designer. Click on the New Extract button on the toolbar. The Select the Text File window will display.

At the Select the Text File window, click on the down arrow next to the data box and navigate to TUTOR4.REP in your \Common800 directory. Click OPEN. The report will open in the Data Panel. The Source Options window will open as well.

2 While in the Source Options window, click on the Display Choices tab. At the display tab, turn Pad Lines ON by clicking on it once to display a check mark in the box. Click OK. From Preferences on the toolbar, make sure Close Definition Dialogs on Add/Update is ON. Now you are ready to begin.

3 The next step is to find the patterns in this file and build recognition patterns (Line Style Definitions) so the Extract Schema Designer can identify the lines with data. The first characteristic of this report to notice is that each data record uses two lines of text. Another important characteristic is that several characters on each line are repeated consistently in the same position. These consistent patterns will make it easy for you to build Line Style Definitions.

a. Highlight a pattern. – Highlight the letters TRN in column 15 through 17.

3-29


The letters TRN are in the same position in the first line of every record in the report. We could use the slash ( / ) in the third column or the colon ( : ) in the ninth column or any of several other consistent characters to identify the line, but the TRN will do. The Extract Schema Designer only needs one consistent characteristic to identify a line.

b. Select Auto New Line Style and specify action. – Right-click in the Data Panel, select Define Line Style from the pop-up menu and select Auto New Line Style from the second level pop-up menu. Then select Collect Fields from the third level pop-up menu.

The Extract Schema Designer will automatically create a Line Style using TRN in column 15 through 17 as the Recognition Pattern and as the Line Style Name. Each line of text that matches this Line Style will now display the Line Style Name TRN and a small (gray barbell) in the Line Style Column to the left of the text.

c. Highlight a pattern. – In the second line of text, highlight the word ATTDOC in column 19 through 24.

These six letters appear in the same position in each second line of every record in the report.

d. Select Auto New Line Style and specify action. – Right-click in the Data Panel, select Define Line Style from the pop-up menu and select Auto New Line Style from the second level pop-up menu. Then select Accept Record from the third level pop-up menu since this is the last line of every record.

The Extract Schema Designer will automatically create a Line Style using ATTDOC in columns 19 through 24 as the Recognition Pattern and ATTDOC as the Line Style Name. Each line of text that matches this Line Style will now display the Line Style Name ATTDOC and a green arrow in the Line Style Column to the left of the text line.

4 First look at the "TRN" lines. Notice that only the first couple of Data Fields in each of those lines falls within the same columns from record to record. Define those fields first:

a. Highlight the data. – In the TRN line, highlight the logged date and time data from column 1 through 13.

3-30


b. Select New Data Field. – Right click in the Data Panel, select Define Data Field from the pop-up menu and select New Data Field from the second level pop-up menu. The Field Definition window will open.

c. Name the field. – The default Field Name is not very descriptive, so overwrite the default by typing Log in the box.

d. Click Add. – Then click on the Add button.

Notice that the date and time data text changes to red in each of the TRN lines within the report and the (gray barbell) to the left changes to . The red in the text indicates that it is the first field on that line. The new barbell indicates that there is at least one data field defined for that line, but the data field is not assigned to an Accept Record.

e. Highlight the data. – Next, highlight the word TRN from column 15 through 17.

f. Select New Data Field. – Select Define Data Field à New Data Field from the pop-up menus. The Field Definition window will open.

g. Name the field. – Overwrite the default field name by typing Trans_Type in the box.

h. Click Add. – Then click Add.

Notice that the TRN text changes to green all through the report indicating that it is the second data field defined on that line.

i. Highlight the data. – Finally, highlight the 12-digit number from column 19 through 30.

j. Select New Data Field. – Select Define Data Field à New Data Field from the pop-up menus. The Field Definition window will open.

k. Name the field. – In the Field Definition window, type Trans_No in the field name box.

l. Click Add. – Click Add.

Notice that the numeric text changes to blue in each of the TRN lines within the report indicating that it is the third field defined on that line.

3-31


OK, that’s it for fixed length, fixed position Data Fields on the TRN lines of text in this report. The remaining Data Fields require some of the special features of the Extract Schema Designer to define.

5 Because the patients’ and doctors’ names are different lengths, you cannot use Fixed Position to define the remainder of the Data Fields on the TRN lines. But because all of the fields other than the names have field labels with colons and spaces, Field Tags, you can define those fields as Floating Tag. (Floating just means that the Field Tags are not in the same position on the line in every record. If there were no Field Tags, you could still define the fields using Relative Word Position.)

The fourth field starts in the same column in each of the TRN lines so you can define this field’s Start Rule as Fixed Position.

Click the Vertical Positioning Bar button and click at the beginning of the field to confirm that the field does indeed start in the same position in every record. Then click the button again to get rid of the red line. The End Rule will be Floating Tag because "TIM:", the tag for the next field, always occurs at the end of this field.

a. Highlight the field: Highlight the patient’s name from column 32 through 47.

b. Select New Data Field: Right click. Select Define Data Field à New Data Field from the pop-up menus. The Field Definition window will open.

c. Name the field: Type Patient in the Field Name box.

d. Set the Start Rule: The Start Rule is correct already, so click on the End Rule tab.

e. Set the End Rule: Click on the Floating Tag option and notice that the cursor is now blinking in the box to the right of the option. Type TIM: in the box. (This tells the Extract Schema Designer that this Data Field ends when the "TIM:" Field Tag is encountered.)

There is one extra step necessary for fields that are not fixed in length, which is setting the Default FldLength to prevent truncation. Set the Default FldLength in bytes for the field; 30 should be about right for this field. This can be done either on the End Rule tab, or the Data Collection/Output tab.

3-32


f. Click Add.

Notice that the patient’s name does not change to colored text in the report. Fields defined as Floating Tag or Relative Word Position will not appear in colored text, nor will they be underlined even if you have Underline Fields turned ON in the Preferences menu. This is because those field positions are not the same in all records.

The remaining fields start and end in different columns in each of the TRN lines (Use the Vertical Positioning Bar to confirm this if you wish.), so you can define these fields’ Start Rules and End Rules as Floating Tag, except the last field.

a. Highlight the field: Highlight the date and time data from column 54 through 71.

b. Select New Data Field: Right Click. Select Define Data Field à New Data Field from the pop-up menus.

c. Name the field: Type Date_Time in the Field Name box.

d. Set the Start Rule: At the Start Rule tab select the Floating Tag option. Type TIM: in the box. (This tells the Extract Schema Designer that this Data Field starts immediately after the text "TIM:".)

e. Set the End Rule: Now click on the End Rule tab. Click on the Floating Tag option. Type TYP: in the box. (This tells the Extract Schema Designer that this Data Field ends when the "TYP:" Field Tag is encountered.)

Click on the Data Collection/Output tab. The Default FldLength for this field should be 18, which is fine. Leave it as is.

f. Click Add.

Repeat the steps above for all except the last field (RATE).

a. Highlight the field.


c. Name the field.

d. Set the Start Rule.

e. Set the End Rule and export length.

f. Click Add.

3-33


6 The RATE field at the end of the TRN line starts with a Floating Tag, but ends at the end of the line of text. So that is how you will define this Data Field. (Use the same 6 steps but a different End Rule.)

a. Highlight the field: Highlight the rate data from column 93 through 97.

b. Select New Data Field: Right click. Select Define Data Field à New Data Field from the pop-up menus.

c. Name the field: Type Rate in the Field Name box.

d. Set the Start Rule: At the Start Rule tab select the Floating Tag option. Type RATE: in the box.

e. Set the End Rule: Click on the End Rule tab. Click on the End of Line option.

Click on the Data Collection/Output tab. Set the Default FldLength in bytes for the field.

f. Click Add.

You have completed all the work that is necessary to extract the data from one line of text. Now proceed to define the Data Fields on the second line of text.

7 Look at the ATTDOC line of text in the records. Notice that the Data Fields in this line are also Floating Tag data. Follow these steps to define all the Data Fields except the last field.

a. Highlight the field: Highlight the attending doctor number from column 29 through 34.

b. Select New Data Field: Right click. Select Define Data Field à New Data Field from the pop-up menus.

c. Name the field: Type Attdoc_No in the Field Name box.

d. Set the Start Rule: At the Start Rule tab select the Floating Tag option. Type ATTDOC NO: in the box.

e. Set the End Rule: Click on the End Rule tab. Click on the Floating Tag option. Type ATTDOC: in the box.


a. Click Add.

Repeat the following basic steps (using the correct field names and tags) for the remainder of the Data Fields on the ATTDOC line, except the last field (BY).

3-34


b. Highlight the field.

c. Select New Data Field.

d. Name the field.

e. Set the Start Rule.

f. Set the End Rule and export field length.


g. Click Add.

8 The BY field at the end of the ATTDOC line starts with a Floating Tag, but ends at the end of the line of text just like the RATE field in the first line. So, use the same steps as before, except use End of Line as the End Rule for that field.

a. Highlight the field.


c. Name the field.

d. Set the Start Rule.

e. Set the End Rule: Click on the End Rule tab. Click on the End of Line option.


f. Click Add.

You have completed all the work that is necessary to extract the data from the second line of text.

9 To verify that you have defined the Line Styles and Data Fields correctly, click on the Browse Data Record button on the tool bar. Since there is only one Accept record, a message window will appear saying, "Fields Assigned to Accept Record ATTDOC." Click OK. There should be five records, each consisting of 14 Data Fields.

Close the browser by clicking the X in the upper right corner.

10 You are now ready to save your newly defined extract script.

Click on the Save Extract button in the toolbar. The Save Extract window will display.

3-35




Enter a description of Tutorial 4 if you wish and click the OK button. Your new extract script is now saved.


You can now close the Tutorial 4 extract script by clicking the Close Extract button in the toolbar.

From here you can exit the Extract Schema Designer by choosing Exit from the File menu.

OR

You may create another extract script. See the other Tutorials for instructions on building different kinds of extract scripts.

3-36

Tutorial 5 - Columnar Data with a Footer

Tutorial 5 - Columnar Data with a FooterTutorial 5 is designed to guide the new user through the steps necessary to create and save a script file in the Extract Schema Designer that reads and flattens a data file that contains both detail lines and a footer line with data to extract.

This tutorial would be useful to anyone likely to be working with columnar data with footer. Before doing this tutorial, it is recommended that you do “Tutorial 3 - Columnar Data” on page 3-20 first.


Procedure

1 Open the Extract Schema Designer. Click on the New Extract button in the toolbar. The Select the Text File window will display.

At the Select the Text File window, click on the down arrow next to the data box and navigate to TUTOR5.REP in your \Common800 directory. Click OPEN. The report will open in the Data Panel. The Source Options window will open as well.

2 While in the Source Options window, click on the Display Choices tab. At the display tab, turn Pad Lines ON by clicking on it once to display a check mark in the box. Click OK. Select Preferences on the toolbar, make sure Close Definition Dialogs on Add/Update is ON. Now you are ready to begin.

3 The primary data record information is in the table detail lines. This data is highly structured in neat consistent columns. The Extract Schema Designer can build recognition patterns for Line Styles and Data Fields with this type of data automatically, saving you a lot of time and effort.

a. Select the line. – Select the first detail line (it begins with SALES/MARKETING) by clicking in the Line Style column immediately to the left of the line to highlight the entire line of text.

3-37


b. Select Parse Columnar Data. – Then right click in the Line Style Column (the yellow stripe on the left part of the screen). From the pop-up menu, select Parse Columnar Data.

The Extract Schema Designer automatically creates a Line Style for the line and gives it a default Line Name of "SALESMARKETING". The Extract Schema Designer also automatically parses the line into 7 Data Fields using spaces as a column separator. The Data Fields will be given default names SALESMARKETING_1 through SALESMARKETING_7.

Notice in the Data Panel that the Data Fields on the line now alternate colors to indicate that they have been defined. Also, if you have Field Underlining turned ON, each Data Field will be underlined with a dashed colored line that designates the width of the Data Field.

4 Since the Data Field names are used in the Browse Data Record as column headings for the Data Fields, it’s a good idea to change the Data Field names for SALES/MARKETING_1 through SALES/MARKETING_7 to be more descriptive field names.

Here are the new, more descriptive names for the Data Fields:

SALES/MARKETING_1 Department






SALES/MARKETING_7 DepartmentTotal

Here are the steps to take to change the Data Field names:

First, so you don’t have to open and close the Field Definition window several times, in the Preferences menu, turn Close Definition Dialogs on Add/Update OFF by clicking on it once with the mouse.

a. Open the Field Definition window. – Double click on one of the Data Fields in the SALESMARKETING line to open the Field Definition window.

3-38


b. Name the Data Field. – In the Field Definition window select the default field name, highlight it, and replace it with the corresponding descriptive name given above.

c. Push Update. – Click the Update button.

d. Select the next Field. – To select the next Data Field, click on the arrow beside the Field Name box and a list of Data Fields that have been defined for the current Line Style will be displayed. Select the next Data Field.

e. Repeat steps 2, 3, and 4. – Name the next Data Field, push Update, and select the next Data Field. Keep going until you have named all the fields.

f. Click Close. – Close the Field Definition window by clicking on the Close button when you have updated all the fields.

5 Let’s change the Line Style name to something more appropriate as well. Double click on the name, SALESMARKETING, in the Line Style column on the left of the screen. The Line Style Definition window will open.

Type in a new name, Detail.

Click Update.

Notice that the Line Style name in the Line Style column on the left of the Script Designer window changes from SalesMarketing to Detail. The icon to the left of the Line Style name is now , indicating that it is a Collect line with defined, but unassigned, data fields.

6 The SALESMARKETING Line Style is recognized by a pattern where columns 2 to 16 contain the text "SALES/MARKETING". This pattern matches only the first detail line. It needs be general enough to match all of the detail lines in the text, but specific enough to match only the detail lines, not the TEAM TOTALS line.

a. Analyze the lines. – Analyze the detail lines to find what makes them unique in comparison to other lines in the text. Things to look for are position of the Data Fields, contents of the Data Fields, anything that is consistent for each of the detail lines but not contained in non-detail lines. For example, the detail lines contain:

Commas in positions 24, 34 and 75 on every line.

3-39


Only letters, white space, and a / in columns 2 through 18.

Only digits, white space, and commas in columns 20 through 79.

A digit in position 78.

An upper case letter in each of the first 5 positions.

Of all of the above observations, creating a pattern to look for uppercase letters in the first 5 positions is the best way to go. Here are some reasons why:

Defining a pattern that would check for commas in positions 24, 34, and 75 would require 3 pattern lines and probably would not match every detail line in subsequent reports. Suppose in this same report (created a week later) Team 2 of the Development department went to a pre-paid weeklong class and they only spent 100 dollars on supplies for the class. This means that a comma would not be in position 34 of that detail line so it would not match the Line Style.

Defining a pattern to check for letters, white space, and a / in columns 2 through 18 would require three pattern lines and also matches the column heading line.

Defining a pattern to match lines that contains at least one digit in positions 20 through 79 and does not contain letters or / would require 3 pattern lines. It also matches the Team Totals line.

Defining a pattern to match lines that contain a digit in position 79 would match the detail lines and the Team Totals line.

So, the best pattern to use is one that looks for capital letters in columns 2 through 6.

b. Open the Line Style Definition window. – The window should already be open so you can skip this step.

c. Define a recognition pattern.

3-40


d. Extend and clarify the recognition pattern. – Now you need to add another row to the Recognition Rule grid. The row that you just defined would pick up the Processing Date line in the footer. And since that line needs its own Line Style, you must further refine the "Detail" Line Style so it does not include the footer line.

Table 3-2 To Define a Recognition Pattern

Steps

1. Open the Pattern Builder window. – In the Line Style Definition window, click once in the Look For? cell in the first row of the grid, and then click the down arrow that appears. This will display the Pattern Builder window.

2. Select the type of data. – Change the value of the Type field from literal to character class by clicking in the Type cell, then clicking the arrow to display the allowable values for the Type field. Select character class from the list.

3. Enter a value. – Change the value of the Value field to upper case letters by clicking in the Value cell, then clicking the arrow to display the allowable values for the Value field. Select upper case letters from the list.

4. Enter the count and location. – Highlight the value in the Count field and change it to 5. The value of the Begin field should be 2. Change the value of the End field to 6 by highlighting the current value in the End cell and typing a 6.

5. Click OK. – Click the OK button in the Pattern Builder window to save the changes and return to the Line Style Definition window.

Table 3-3 To Refine the Detail Line Style

Steps

1. Add another line. – Click in the first empty cell in the And/Or column. The word And will automatically display in that cell.

2. Select search parameters. – Now click in the first empty cell in the Search What? Column. Then click on the down arrow and select Column Range (m-n) from the list.

3. Select operator. – Click in the first empty cell in the Operator column. Then click on the down arrow and select Does Not Contain from the list.

4. Define a recognition pattern.

3-41


e. Click Update. – Click on the Update button in the Line Style Definition window. (Notice that "Detail" now appears beside each of the detail lines in the Line Style column, and not next to the Processing Date line.)

f. Click the Close button to close the Line Style Definition window.

7 Now it’s time to modify Data Fields in the Detail lines so the Extract Schema Designer will be able to extract the data on the last line.

a. Open Field Definition window. – Double-click in the Department Data Field, the red field at the beginning of each detail line. The Field Definition window will open.

b. Select the Array Fields option. – Click on the Data Collection/Output Options tab, and click on the Array Field option to turn it ON.

c. Click Update. – Click the Update button

d. Repeat for all fields. – Click on the down arrow next to the Data Field Name box and choose the next Data Field.

Repeat this process for each field in any one line of text. Since these fields exist in each of the Detail lines, this modification affects all of these lines.

Table 3-4 To Define a Recognition Pattern

Steps

1. Open the Pattern Builder window. – Click in the first empty cell in the Look For? column. Then click on the down arrow. This will open the Pattern Builder window.

2. Select the type of data. – In the Pattern Builder window, the Type column will default to literal. That’s what you want here, so leave the default.

3. Enter a value. – If the Value column is not empty, delete the contents of that cell. Press <TAB> or place the mouse pointer in the cell in the Value column and click once to position the blinking cursor in that cell. Type a capital P.

4. Enter the count and location. – Change the values in the Count, Begin and End cells all to 1.

5. Click OK. – Then click on OK in the Pattern Builder window.

3-42


8 To define the PROCESSING DATE line, highlight the text PROCESSING DATE: with the mouse.

a. Highlight a pattern. – Right-click in the Data Panel, the large white area of the screen.

b. Select Auto New Line Style and specify line action. – Choose Define Line Style from the pop-up menu and Auto New Line Style from the second. From the third level pop-up menu, choose Accept Record.

The Line Style Definition window will not open, because the Extract Schema Designer has done everything automatically. The Extract Schema Designer has defined the Line Style using the first word of the highlighted text in that position as the recognition pattern and named the Line Style PROCESSING.

Notice that PROCESSING_DATE and a green arrow has appeared in the Line Style column to the left of that line.

9 Now, define the Data Field on the PROCESSING_DATE line.

a. Highlight the data. – Highlight the date from column 17 through 24.

b. Select New Data Field. – Then right-click the mouse in the Data Panel and choose Define Data Field and New Data Field from the pop-up menus.

c. Name the field. – When the Field Definition window opens, change the default Field Name to Date.

d. Click Add. – Then click the Add button in the Field Definition window.

Then Close the window. Notice that the icon for the PROCESSING_DATE line style has changed. It is now , indicating that it is an Accept line with defined, but unassigned, data fields.

10 Click the Browse Data Record button in the Tool. Since there is only one Accept record, a message window will appear saying, "Fields Assigned to Accept Record PROCESSING_DATE." Click OK.

Click the X in the upper right corner to close the Source Browser.

11 To reorder the data fields, choose the Export Field Layout option from the Field Menu. A list of Data Field names and positions will display.

3-43


Move the Data Field names by pressing and holding down the left mouse button while dragging each Data Field to its desired position on the list. When you release the mouse button the Data Field should be repositioned in the first position.

Note The dragged Data Field will be inserted above the Data Field you are positioned on when you release the mouse button.

Click OK to save these changes.

12 Click the Browse Data Record button again. Notice the Data Fields are now in their desired positions. Close the window.







You can now close the Tutorial 5 extract script by clicking the Close Extract button in the Tool Bar.


OR


3-44

Tutorial 6 - Variable Length Multi Line Data Fields

Tutorial 6 - Variable Length Multi Line Data FieldsTutorial 6 is designed to guide the new user to create a script that reads and flattens a data file that contains data that extends across multiple lines of text and where the end of each record varies.

This tutorial will be useful to anyone who has a report with fields that extend across more than one line, or has no consistent end of record line.


Procedure

1 Open the Extract Schema Designer. Click the New Extract button in the toolbar. The Select the Text File window will display.

2 At the Select the Text File window, click on the down arrow next to the data box and navigate to TUTOR6.REP in your \Common800 directory. Click OPEN. The report will open in the Data Panel. The Source Options window will open as well.

3 While in the Source Options window, click on the Display Choices tab. At the display tab, turn Pad Lines ON by clicking on it once to display a check mark in the box. Click OK. Go to Preferences on the toolbar, make sure Close Definition Dialogs on Add/Update is ON. Now you are ready to begin.

4 Scroll through the data to get an idea of its structure. Notice that there are eight (8) or nine (9) left-aligned Field Tags in each record. These tags can be used to easily identify and define the Line Styles in this report.

a. Highlight a pattern. – In the third line, highlight DATE: from column 1 through column 5.

3-45


b. Select Auto New Line Style and specify action. – Right click with the mouse positioned anywhere in the Data Panel (the white part of the screen) and select Define Line Style from the first level pop-up menu. Select Auto New Line Style from the second level pop-up menu and Collect Fields from the third level pop-up menu.

The Extract Schema Designer will automatically define the Line Style with a Recognition Pattern of "DATE:" in columns 1 through 5 and name the Line Style DATE.

A gray barbell icon will appear to the left of the Line Style name, indicating that the line has no defined Data Fields.

5 Repeat this same basic procedure in step 4 for each of the following Field Tags in the first record.

a. Highlight a pattern.

b. Select Auto New Line Style and specify action.

Field Tag Columns

RECORDATION: 1 - 12

CONSIDERATION: 1 - 14

SITE DIMENSIONS: 1 - 16

SITE AREA: 1 - 10

ZONING: 1 - 7

REMARKS: 1 - 8

(You may also, if you wish, go to the second record and define the LEGAL DESCRIPTION: field in columns 1-18.)

6 Notice we did not include the Field Tag UNIT PRICE in that group. Since the UNIT PRICE data is the last section of text for each record in this report, you need to define it using the ACCEPT Record Line Action. Proceed as follows:

a. Highlight a pattern. – In the 23rd line of the report use the mouse to highlight UNIT PRICE: from column 1 through column 11.

b. Select Auto New Line Style and specify action. – Right click with the mouse positioned anywhere in the Data Panel and select Define Line Style and Auto New Line Style from the pop-up menus. This time, select Accept Record from the third level pop-up menu.

3-46


The Extract Schema Designer will automatically define the Line Style with a Recognition Pattern of "UNIT PRICE:" in columns 1 through 11 and name the Line Style UNIT_PRICE.

A green arrow icon will appear next to the Line Style, indicating that this is an Accept line with no defined Data Fields.

To verify that the Line Style Definitions match the appropriate lines of text throughout the report, you can scroll down and see that each of the lines that contain a Field Tag has the corresponding Line Style Name in the Line Style Column to the left of the text line.

Now scroll back to the top of the report and you can begin defining the Data Fields.

7 Define the first data field as follows:

a. Highlight the data. – Highlight November 12, 1993 from column 26 through column 42.

b. Select New Data Field. – Right click. Select Define Data Field àNew Data Field from the pop-up menus. The Field Definition window will open.

c. Name the Data Field. – The Field Name will default to DATE_1. Type DATE to overwrite the default or click with the mouse in the Field Name box and backspace over the _1.

d. Adjust Settings. – All the other settings in the tabs are correct for this Data Field.

e. Click Add. – Click the Add button to add this field to the script.

The highlighted data will change to red text, and if you have Underline Fields turned ON in the Preferences menu, it will be underlined with a dashed red line.

The gray barbell will change, indicating that the Collect line now has a defined, but unassigned, Data Field.

8 Now define the Data Field in the RECORDATION line as follows:

a. Highlight the data. – Highlight Jeff County from column 26 through column 36.

b. Select New Data Field. – Right click. Select Define Data Field à New Data Field in the pop-up menus.

3-47


c. Name the Data Field. – The Field Name will default to RECORDATION_1. Type in something else if you wish or click with the mouse and backspace twice to remove _1.

d. Adjust Settings. – All the other settings in the tabs are correct for this Data Field.

e. Click Add. – Click the Add button to add this field to the script.

9 Notice that some of the data you want to extract resides within a single line of text in one record but continues across multiple lines of text in other records. For example, the data in the CONSIDERATION field in the first record is on a single line of text, but in the second record, the data in the CONSIDERATION field continues across nine (9) lines of text. This can be easily defined using the Extract Schema Designer’s feature called Continuation Rule.

a. Highlight the data. – Highlight $333,000 Cash from column 26 through column 38.

b. Select New Data Field. – Right click. Select Define Data Field New Data Field in the pop-up menus.

c. Name the Data Field. – Change the field name if you wish.

d. Adjust Settings. – The Start Rule is correct, so leave the default setting as is. Click on the End Rule tab. Select the End of Line option. Click on the Continuation Rule tab. Select the Until Next Line Style option.

There is one extra step necessary for fields that are not fixed in length, which is setting the Default FldLength to prevent truncation. Set the Default FldLength in bytes for the field; 500 should be about right for this field. This can be done on either the End Rule tab or the Data Collection/Output tab.

Notice the Rectangular Continuation option toward the bottom of the Continuation tab. The default is ON. This is correct for this particular data because the text in this Data Field doesn’t wrap back to the left margin (column 1) in the report. It only wraps back to column 26 where the first line of text in this field begins. So leave this box checked.

e. Click Add. – Click on the Add button to add this Data Field to the script.

3-48


None of the data in this Data Field will change to colored text because the End Rule was set to End of Line. Since the rule is not a set column, the Extract Schema Designer cannot paint it a color. Its location may vary from one record to the next.

If you are not sure that the Extract Schema Designer is getting the whole field, you can verify that the Extract Schema Designer is picking up all the data by clicking the Browse Data Record button, and widening the CONSIDERATION column. Also, if you hover the mouse pointer over the cell you want to see, the Extract Schema Designer will display a tool tip box to show you the cell’s contents.

10 Define all of the remaining Data Fields using the basic procedure in Step # 9 above.



c. Name the Data Field.

d. Adjust Settings.

e. Click Add.

11 When you have defined all eight (or nine) of the Data Fields, click on the Browse Data Record button. Since there is only one Accept record, a message window will appear saying, "Fields Assigned to Accept Record UNIT_PRICE." Click OK. You may now confirm that you have six (6) records and that each record contains eight (or nine) Data Fields and that all the data from the long fields was collected.





3-49




You can now close the Tutorial 6 extract script by clicking the Close Extract button in the toolbar.


OR


3-50

Tutorial 7 - Multiple Accept Records

Tutorial 7 - Multiple Accept RecordsTutorial 7 is designed to guide the new user through the steps necessary to create and save an extract script file in Extract Schema Designer that utilizes multiple Accept Records.

You will parse the data in a report file, TUTOR7 (supplied during installation), into a format suitable for exporting via the Map Designer into a multiple record file. By following the steps outlined below, you will become familiar with both the process of creating an extract script and the terms used throughout the documentation.

Procedure

1 From the Extract Manager window, open a New Extract by clicking on the New Extract button in the toolbar. The Select the Text File window will display.

2 At the Select the Text File window, click on the down arrow next to the data box and navigate to TUTOR7.TXT in your default installation directory (...\Common800). Click Open and the report file will load and display in the Data Panel (the large white space in the Extract Designer window). The Source Options window will open as well.

3 While in the Source Options window, click on the Display Choices tab. At the display tab, turn Pad Lines ON by clicking on it once to display a check mark in the box. Click OK. Go to Preferences on the toolbar, make sure Close Definition Dialogs on Add/Update is ON. Now you are ready to begin.

4 The first thing to do with this file is to begin creating Line Styles for the first Accept Record. To create a Line Style and Data Fields for these lines:

a. Select the line. – Select the first detail line (it begins with Parmer Lane Animal Hospital) by clicking in the Line Style column (the narrow yellow stripe on the left) immediately to the left of the line. This should highlight the entire line of text.

b. Right click anywhere in the Line Style column and select New Line Style from the pop up list.

A Line Style Definition window will appear.

3-51


c. The Line Style Name defaults to Parmer_Lane_Animal_H. Go ahead and give it a new name, HospitalLine. The rest of the information for this Line Style is correct.

d. Click Add.

Notice that a gray barbell appears to the left of HospitalLine. This indicates that there is no information defined for that line style to collect yet.

e. Now highlight Parmer Lane Animal Hospital in the Data Panel (the large white space in the Script Designer window).

f. Right click anywhere in the Data Panel and select Define Data Field from the pop up list.

g. Select New Data Field from the second pop up list.

A Extract Schema Designer Field Definition window will appear.

h. The Field Name defaults to HospitalLine_1. Change it to Hospital. The remainder of the information is correct.

i. Click Add.

Notice that the barbell to the left of the HospitalLine Line Style name is now colored, indicating that it now has information to collect, but the data field is unassigned.

5 Click in the Line Style column to the left of April 1, 1999.

6 Right click anywhere within the Line Style column and select New Line Style from the pop up list.


7 Change the Line Style Name to ReportDateLine. Click Add.

The data on this line will always be centered beneath the HospitalLine. Depending on what month and day the report is run on, the data field may be longer or shorter than the current date. You need to reflect this when defining your data field.

8 To make sure that your data field is wide enough, highlight the data from position 31 to position 57 (extending beyond the end of line symbol). Now right click anywhere in the Data Panel and select Define Data Field from the pop up list.

9 Select New Data Field from the second pop up list.

A Field Definition window will appear.

10 Change the field name to ReportDate.

3-52


11 Click the Data Collection/Output tab. Since you will not want leading or trailing spaces in your export data, you should make sure that the Trim Leading and Trailing Spaces box under Other Collection Options is checked.

12 Click Add.

13 The first repeating Line Style that we want the Extract Schema Designer to find is the Account Number. Click in the Line Style column to the left of 1101-01. This will select the entire line.

14 Right click anywhere in the Line Style column and select New Line Style from the pop up list.

15 The Line Style Name will default to STYLE1. Change it to AccountLine.

Notice the entry under the Look For? column. Its default is "-" in position 5. While this does catch all pertinent lines in our sample, it might not catch all instances in a larger record sample. Therefore, you need to update the Line Recognition Rules.

16 Click in the first cell under Look For. A Pattern Builder window will open.

17 The Type defaults to literal. What we want is a mask, because it’s more specific than simply looking for a character class. So click in the first cell under Type and select Mask from the drop down list.

18 Now you need to tell the Extract Schema Designer the exact pattern you are looking for. To do this, click in the first cell under Value. Delete the hyphen that is already there, and type:

This tells the Extract Schema Designer that there will be four digits followed by a hyphen, and then two more digits.

19 Change the Begin position from 5 to 1.

20 Change the End position from 5 to 7.

21 Click OK.

22 Click Add.

A gray barbell will appear to the left of AccountLine. You can scroll through the data to make sure that the Extract Schema Designer now recognizes all account number lines.

3-53


23 Now highlight the account number (1101-01) and right click anywhere in the Data Panel.

24 Select Define Data Field from the pop up list.

25 Select New Data Field from the second pop up list.


26 Change the Field Name from AccountLine_1 to AccountNo.

27 Click Add.

The barbell to the left of the AccountLine Line Style has changed, indicating that there is a defined, but unassigned, Data Field. Note that the account number appears at the beginning of each subsequent record. Therefore, we can use this to determine the position of the untagged lines beneath it.

28 Before you continue, go up to Source Options and click the box next to Flush Field Contents on Accept Default under the Script Design Choices tab. This will flush the data from the remaining fields in your report, unless you manually change a field to propagate. You are now ready to proceed with the remainder of the file.

29 Select the first detail line under the first account number. Click in the Line Style column to the left of Robertson.

30 Right click in the Line Style column.

31 Select New Line Style from the pop up list.


32 Rename the Line Style from Robertson to LastNameLine.

33 Now click on the down arrow to the right of the Recognized By box.

34 Select Relative Position from the drop down list that appears.

Note the change in the information under Line Recognition Rules. The default Base Line should be AccountLine. If it is not, click in the first cell under Base Line and select Account Line from the drop down list that appears.

35 The default Line Count from Account Line is 1. However, there is a blank line between the AccountLine and the LastNameLine. So change the count from 1 to 2.

3-54


36 Click Add.

A gray barbell appears to the left of LastNameLine.

37 Now highlight Robertson and several places beyond the end of line, in case someone further in the file has a very long last name. Highlighting out to position 35 should do it.

38 Right click in the Data Panel.


40 Select New Data Field from the pop up list.


41 Change LastNameLine_1 to LastName.

42 Click Add.

43 Click in the Line Style column to the left of Linda.

44 Right click anywhere in the Line Style column.



46 Change the Line Style name from Linda to FirstNameLine.

47 Click on the down arrow to the right of the Recognize By box. Select Relative Position from the drop down list that appears.

48 The Line Recognition Rules should default to 3 lines from AccountLine. If it does not, change the Count and the Base Line accordingly.

49 Click Add.

50 Now highlight Linda and several places beyond the end of line, in case someone further in the file has a very long first. Highlighting out to position 35 should do it.





54 Change FirstNameLine_1 to FirstName.

55 Click Add.

3-55


56 Click in the Line Style column to the left of 143 Patterson Place.



A Extract Schema Designer Line Style Definition window will appear.

59 Change the Line Style name to Address1Line.



62 Click Add.

63 Now highlight 143 Patterson Place and several places beyond the end of line, in case someone further in the file has a very long first. Highlighting out to position 60 should do it.





67 Change Address1Line_1 to Address1.

68 Click Add.

69 Click in the Line Style column to the left of Austin TX 78759.




72 Change the Line Style name to CSZ (for City, State, Zip).



3-56


75 Since this is the end of the first record (before the information becomes a subset of this information), you need to make this line an Accept Record. Click on the Line Action tab and select Accept Record Including.

76 Click Add.

Notice that a green arrow appears to the left of the CSZ Line Style.

77 Right-click the Line Style column again.

78 Select Parse Columnar Data from the pop up list.

Notice that the fields to the right of CSZ are now underlined in three different colors. They are now separate fields of CSZ. Also notice that the green arrow to the left of the CSZ Line Style has changed, indicating that it is collecting information before it completes the record, but the Data Field is not assigned to an Accept Record yet.

79 Double-click the red Austin field.

A Field Definition will appear.

80 Rename the Field Name from CSZ_1 to City.

81 Click Update.

82 Double-click on the green TX field.

A Extract Schema Designer Field Definition window will appear.

83 Rename the Field Name from CSZ_2 to State.

84 Since this field will always be a two-place field, click on the End Rule tab and change the Fixed Column number to 23.

85 Click Update.

Notice that State field now only has the two letters underlined in green.

86 Double-click on the blue 78759 field.


87 Rename the Field Name from CSZ_3 to Zip.

88 Since this field will always start at position 40, change the Start Rule Fixed Column accordingly.

3-57


89 Since this field may go out to position 49, change the End Rule Fixed Column accordingly. If you do not extend this field, the last four digits in the zip for the second record will not be picked up.

90 Click Update.

91 To view the data that you’ve collected so far, click on the Browse Data Record button.

92 Since there is only one Accept record, a message window will appear saying, "Fields Assigned to Accept Record CSZLine." Click OK. All of the Data Fields will appear in the Browse Data Record window. This is where you can preview how your data will look in a normal database format and check to see that you have defined everything correctly.

93 Close the Browse Data Record by clicking the X in the upper right corner.

Once you have assigned your data fields to an Accept record, the icons next to your Collect lines will change to , and the icon next to your Accept line will change to . The blue in the barbells indicates that the data field has been assigned to an Accept record.

Since the fields in the Browse Data Record window are not in the order you wish them to appear, go to Field in the toolbar and choose Export Field Layout from the drop down list that appears.

To move the fields to their desired positions, press and hold the left mouse button on the Field Name.

While still holding down the left mouse button drag each Field Name to the desired location on the list and release the mouse button. You will see a special symbol ( ) while you are dragging the field name to its proper place.

You may now reopen the Browse Data Record window (click on the icon) to view your export fields as they will appear in your database format.

94 Next you need to define a line style that will recognize each of the pet information lines. The first step to doing this is to analyze each of these records, to look for a common pattern to define the Line Style.

3-58


The Pet Type, Pet Name, Sex, and Color will all be different in each record. The only thing that remains the same is the placement of Age, in position 42 and 43.

To begin defining your Line Style to recognize this pattern, click in the Line Style column to the left of DSH-C.

95 Right-click anywhere in the Line Style column.



97 Change the Line Style Name from DSHC to PetInfoLine.

98 Click in the first cell under Look For.

99 Click on the down arrow that appears to the right of that cell.

A Pattern Builder window will appear.

100 Since you don’t want to look for a specific character change the Type from literal to character class.

101 Now that the Extract Schema Designer is looking for a character class, you need to change the Value to something more generic. Click in the first cell under Value and select digits from the drop down list.

102 Since you can’t be sure whether there will be one or two digits for the Age, change the Count from 5 to 1.

103 Now change the Begin position from 3 to 42, the first possible Age position.

104 Then change the End position from 7 to 43, the last possible Age position.

105 Click OK.

106 Since the PetInfoLine may have visit information under it, it will be an Accept Record as well. Click on the Line Action tab and select Accept Record Including.

107 Click Add. Then scroll through the file to see if the Extract Schema Designer now recognizes all of the PetInfoLines. You should see a green arrow to the left of each of lines.

108 With the PetInfoLine still highlighted, right click in the Line Style column again.


3-59


Note that each of the five data fields on that line are now underlined with a different color, and the green arrow to the left of the PetInfoLines has changed, indicating that it has data fields to collect, but the data fields have not been assigned to an Accept Record yet.

110 Since the data fields default to PetInfoLine_1 – PetInfoLine-5, you should go up to Preferences and uncheck Close Definition Dialogs on Add/Update. Then rename each of the fields more appropriately. Click the down arrow next to the Field Name box, and select each new data field. If you make a change to a data field, be sure to click the Update button before moving on to the next data field.

111 Note that the PetAge field is far too large, and does not extend far enough to catch the second digit of the age. Open the data field definition window and change the Start Rule to 42, and the End Rule to 43.

112 Click Update. The field is now properly aligned.

113 Similarly, the PetsSex field will always be a single character. So open the data field definition window and change the End Rule to 58.

114 Click Update. The field is now limited to a single character field.

115 Conversely, the PetColor field needs to be widened to make sure that it will collect longer fields. Open the data field definition window and change the End Rule to 90. That should collect longer descriptions.

116 Click Update. All the PetColor fields should now be fully collected. You should now return to the Preferences menu and reclick the Close Definition Dialogs on Add/Update.

117 Since there may be office visit records for each pet, you should propagate the pet name so that it can be used with the visit date. To change the Behavior on Accept, double-click on Shiva to open the Field Definition window.

118 Click on the Data Collection/Output tab.

119 Click Propagate Field Contents.

120 Click Update.

3-60


121 To view the data that you’ve collected for this Accept Record, click on the Browse Data Record button.

122 Click on the PetInfoLine in the middle of the screen to select the Current Accept Record.

123 Then click on the large button to the right that says Assign to Current Accept Record.

Scroll through the file to see that each pet’s information shows up under the appropriate owner. Click on the X in the upper right corner of the window to return to your script.

124 The last Line Style that you need to define is VisitInfo. To do this, click to the left of 02/18/99 on line 20.

125 Right-click anywhere in the Line Style column and select New Line Style from the pop up list.

126 Change the default NA Line Style Name to VisitInfo.

127 Notice that the Extract Schema Designer is basing its recognition pattern on the two hyphens in the date. Since those characters appear in the same place on each VisitInfoLine, this is fine. However, since this line also needs to be an Accept Record, click on the Line Action tab and select ACCEPT Record Including.

128 Click Add.

A green arrow appears to the left of the VisitInfoLine.

129 With the VisitInfoLine still highlighted, right click in the Line Style column again.


Each of the three data fields on that line are now underlined with a different color, and the green arrow to the left of the VisitInfoLines has changed. This indicates it has data fields to collect, but data fields are not yet assigned to an Accept Record.

131 Since the data fields default to VisitInfoLine_1 – PetInfoLine-3, you should go up to Preferences and uncheck Close Definition Dialogs on Add/Update. Then rename each of the fields more appropriately. Click the down arrow next to the Field Name box, and select each new data field. If you make a change to a data field, be sure to click the Update button before moving on to the next data field.

3-61


132 Since the VisitDate field will always be a fixed length, change the End Rule to 15.

133 Change the End Rule on the Service field to 105 to be sure that it collects all the information in that field.

134 Go back up to Preferences and recheck Close Definition Dialogs on Add/Update.

135 To view the data that you’ve collected for this Accept Record, click on the Browse Data Record button.

136 Click on the VisitInfoLine in the middle of the screen to select the Current Accept Record.

137 Then click on the large button to the right that says Assign to Current Accept Record.

Scroll through the file to see that the visit information shows up under the pet name. Click on the X in the upper right corner of the window to return to your script.

138 There are a few more things that should be done to your script to make the output more user friendly. To start, you should assign the account number to each pet name, and associate the pet name and account number with each office visit. To begin doing this, click on Record in the top toolbar, and select Edit Accept Record.

139 Select PetInfoLine as the Current Accept Record. Then click the Show Fields button in the upper right portion of the Accept Record Definition window.

Notice that there are check boxes beside each field listed. The fields already assigned to the PetInfoLine are checked.

140 Since you want to add the account number to this Accept Record, check the box next to AccountLine.

141 Click Update.

142 You may browse the data to see that the account number now shows up at the bottom of the PetInfoLine fields. Close out of the browser and return to your script window.

143 Click on Record in the top toolbar again, and select Edit Accept Record.

3-62


144 Select VisitInfoLine as the Current Accept Record. Then click the Show Fields button in the upper right portion of the Accept Record Definition window.

Notice that there are check boxes beside each field listed. The fields already assigned to the VisitInfoLine are checked.

145 Since you want to add the account number to this Accept Record, check the box next to AccountLine.

146 You also want to add the pet’s name to this Accept Record, so check the box next to PetName.

147 Click Update.

148 You may browse the data again to see that the account number and pet name now shows up at the bottom of the PetInfoLine fields. Close out of the browser and return to your script window.

149 The final thing that should be done, for the sake of organization, is to rearrange how the fields are displayed for each line style. To do this, click on Field in the top toolbar, and select Export Field Layout.

150 Select PetInfoLine as the Accept Record.

151 To move AccountNo to the beginning of the field list, click and drag AccountNo to the top of the list. Release the mouse. AccountNo should now be the first field of the PetInfoLine.

152 Also, to make it more readable, you should move PetName to the second position in the field list.

153 Click OK.

154 Click on Field in the top toolbar again, and select Export Field Layout.

155 Select VisitInfoLine as the Accept Record.

156 To move AccountNo to the beginning of the field list, click and drag AccountNo to the top of the list. Release the mouse. AccountNo should now be the first field of the VisitInfoLine.

157 Also, to make it more readable, you should move PetName to the second position in the field list.

158 Click OK.

159 Browse your data one more time to see the final layout.

3-63


You are now ready to save your newly defined extract script. To save an extract script:

Click Save Extract in the toolbar. The Save Extract window will display.

Navigate to your default installation directory (...\Common800) and name the extract script file Tutor7. Notice that the Extract Schema Designer suffix is DJP.


Enter a description of Extract Schema Designer Tutorial 7 and click OK. Your new extract script is now saved. Congratulations, you have just created and saved your Tutorial 7 extract script in the Extract Schema Designer!

3-64

Index

AAppend 1-15

BBuild Schema button 2-15

CClearMapPutRecord 1-6create a structure 1-18

DData Change Monitor 1-37Data Parser - Tutorial 4 for Structured Schema

Designer 1-18Decision (properties) window 2-4Decision icon 2-4, 2-14

FFactory Connections tab 1-3Fixed ASCII file to Lotus 1-2-3 r4 file 1-8Function icon 2-15

IIntroduction to Map Designer Tutorials 1-2Introduction to Process Designer Tutorials 2-2

LLoops and Queues 2-11Loops and Queues - Tutorial 2 for Process Designer

2-11

MMap Designer Tutorial 1 - The Basics 1-3Map Designer Tutorial 2 - Parsing an ASCII File 1-8Map Designer Tutorial 3 - Output to an Existing

Target File 1-14Map Designer Tutorial 5 - Single-record Type File to

a Multi-record Type XML File 1-30Map Designer Tutorials 1-2Multiple Accept Records 3-51

NNo event handlers were defined 1-17

OOutput to an Existing Target File 1-14Output to an Existing Target File - Tutorial 3 for Map

Designer 1-14

PParse Data 1-19, 1-27Parsing an ASCII File 1-8Parsing an ASCII File - Tutorial 2 for Map Designer

1-8Process Designer Tutorial 1 - The Basics 2-3Process Designer Tutorial 2 - Loops and Queues 2-

11Process Designer Tutorials 2-2

QQueue icon 2-13, 2-17Queue step 2-13, 2-17

RRepository Explorer 1-3, 1-30, 2-3, 2-11

SScripting icon 2-3Scripting Properties window 2-4Select Connection 1-3Single-record Type File to a Multi-record Type XML

File - Tutorial 5 for Map Designer 1-30Single-record Type File to a Multi-record XML File

1-30Source Connection 1-3Source Data Browser 1-4, 1-9Source File/URI box 1-3Start and stop steps 2-3Starting offset and record length 1-26StartOffset 1-19Step Result Wizard 2-14Stuctured Schema Designer Tutorial 4 - Using the

Data Parser 1-18

Index xli

TThe Basics - Tutorial 1 for Process Designer 2-3The Basics in Map Designer 1-3The Basics in Process Designer 2-3Transformation step 2-15Troubleshooting 1-28Tutorial 1 - Extract Schema Designer 3-4Tutorial 1 - Map Designer 1-3Tutorial 1 - Process Designer 2-3Tutorial 2 - Extract Schema Designer 3-11Tutorial 2 - Map Designer 1-8Tutorial 2 - Process Designer 2-11Tutorial 3 - Extract Schema Designer 3-20Tutorial 3 - Map Designer 1-14Tutorial 4 - Stuctured Schema Designer 1-18Tutorial 5 - Extract Schema Designer 3-37Tutorial 5 - Map Designer 1-30Tutorial 6 - Extract Schema Designer 3-45Tutorial 7 - Extract Schema Designer 3-51

UUse a Structured Schema 1-24Using the Data Parser 1-18

XXML Multirecord Type Target File 1-30

ZZoned Decimal 1-21

xlii Index

Pervasive Tutorial

Documents

microfocus

bit ieee floating

narrow yellow

paid weeklong

yellow area

extract schema

structured

underline