Load Profile Training. Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Agenda Day.

Load Profile Training

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces.

Agenda – Day One

Introductions Review of training goals Review of Innovative Record Structure Anatomy of a Data Load Load Tables (m2btab) Review of standard load profiles Global Variable Functions (@)


Agenda – Day Two

Questions from day one Special Processing Functions (%) Command Functions (#) Translation Tables (m2bmap) Data analysis The m.marcload.local file Test loading records Review and support from Innovative


Training Goals

Understand the database structure Understand existing load profiles Be familiar with issues to consider for

new load profiles Update and create load tables (m2btab) Create translation tables (m2bmap) Add menu options for loading records Test load profiles before production


Topics Not Covered in This Training

Text to MARC conversion programs Loading non-MARC records Loading Contact, Course, Checkin

Cards, Invoice, License, Program, Resource, Session or Vendor records

Complex data manipulation Export tables


Record Structure

Types of records CSDirect @ csdirect.iii.com (see chart) Innovative Guide and Reference no.

101113 The Getting Started Manual @

gsm.iii.com


Each Record Type Contains:

Variable-Length Fields M>I>C>T and M>I>C>M

(Page 7 and IGR page 105783) Fixed-Length Fields

(Page 14 and IGR page 105775)


Anatomy of a Data Load

Load Table(m2btab)

Loading prog.(marc2inn)

Trans. File(bibtemp)

Holding symboltable

m.marcload.local

Data Exchange Load Interface

Translation Table(m2bmap)

Fixed-length fields

Fixed-lengthvalues/codes

Variable-lengthfields

RecordTemplates

Indexing prog.(control)

Indexes

Database

Index rules

Hi/Lo watermark


Load Tables (m2btab)

Control how records are loaded Fields in the incoming record that do not

match load table criteria will not be loaded Create separate load tables for different

loading requirements Data must be in MARC format


Standard Load Tables

Authority records m2btab.a (p. 140) m2btab.anam (p. 142) m2btab.asub (p. 144)

Bibliographic and Item records m2btab.batch (p. 130) m2btab.lti (p. 137)


Standard Load Tables

Bibliographic and Order records m2btab.order (p. 133)

Patron records m2btab.p (p. 146)


Overview of Load Tables

Commenting out lines with “#” Twelve data elements Global Variable Functions (@) Special Processing Functions (%) Command Functions (#) “#” has 2 meanings in load tables


Comment Lines in Load Tables

Any line that begins with “#” is a Comment

Include a brief summary of load table, identify the author and creation date

It is strongly recommended that you add a Comment to every load table you create

Commented lines are not read by the loading program


Element 1 – MARC Tag 690|699|+|0|0|b|d|0|y|N|0|%first=“y”

MARC tags and indicators to be loaded Can be set as a regular expression Can load from any MARC field and the

Leader


Element 2 – New MARC Tag 690|699|+|0|0|b|d|0|y|N|0|%first=“y”

Re-map the data from the tag in element one to the new MARC tag in element two

Cannot be a regular expression If indicators are specified, all indicators

must be present


Element 3 – Subfields 690|699|+|0|0|b|d|0|y|N|0|%first=“y”

Indicates which subfields to load from MARC field in Element 1, MARC

tag Unless specified, subfields are loaded in

the order that they appear in the incoming data


Element 4 - Offset 008|| |35|3|b| |24|n|N|0|lang(b) 690|699|+|0|0|b|d|0|y|N|0|%first=“y”

Used for loading data from positional fields, e.g. Leader, 006, 007, 008

Counting begins at 0 Loading data from single subfield, refers

to beginning of subfield Must be set to “0” when loading the

entire field (usual setting)


Element 5 – Byte Count 008|| |35|3|b| |24|n|N|0|lang(b) 690|699|+|0|0|b|d|0|y|N|0|%first=“y”

Number of bytes to load starting from offset in Element 4

Counting begins at 1 Must be set to “0” when loading the

entire field (usual setting)


Example of MARC 008 field

820305s1991####nyu###########001#0#eng#

Language code at byte 35 of 008 field:

Offset = 35

Byte count = 3


Element 6 – Record Type 690|699|+|0|0|b|d|0|y|N|0|%first=“y”

Determines the record type to load the MARC data into for this line This can be set to:

“b” (Bibliographic) or “i” (Item) or “o” (Order) or “c” (Holdings/Checkin) or “a” (Authority) or “p” (Patron)

Will always correspond with Element 11, Pass Indicator


Element 7 – Variable Field Group Tag 690|699|+|0|0|b|d|0|y|N|0|%first=“y”

Indicates the variable field group tag to load data into

Each line can map to a variable field or a fixed field, but not both

This should be set to “space” if using the line to load a fixed-length field


Element 8 – Fixed Field No. 690|699|+|0|0|b|d|0|y|N|0|%first=“y”

Indicates the fixed-length field to load data into

Fixed-length fields must be referenced by line numbers. See page 14

If the line is used to load a variable-length field then this element should be “0”


Element 9 – MARC/Non-MARC Indicator 690|699|+|0|0|b|d|0|y|N|0|%first=“y”

Determines if the incoming field is loaded in MARC format or not

Works in conjunction with the @marc trigger

Load lines for fixed-length fields always have a non-MARC indicator, “n”

Variable fields usually have a MARC indicator, “y”


Element 10 - Permanence 690|699|+|0|0|b|d|0|y|N|0|%first=“y”

Used for loading during Interactive hardwired Serial Port Interface

Determines whether that setting is retained or not for subsequent loads

Usually set to “N” for most lines in a load table since Serial Port Interface is not used much


Element 11 – Pass Indicator 690|699|+|0|0|b|d|0|y|N|0|%first=“y”

Determines at which pass the line is read during the load

Will always correspond with Element 6, Record Type

Each new record type loaded requires a new pass

Primary record type is always pass 0


Element 12 – Function or Note 690|699|+|0|0|b|d|0|y|N|0|%first=“y”

Global Variable Function (@) Command Function (#) Special Processing Function (%) Text to be inserted Any other beginning character is a

comment and is not read by the loading program


Training Exercise 1

“Creating and Evaluating Load Table Entries”

Turn to page 120 and complete training exercise 1


Global Variable Functionsaka Triggers (@)

Specified in 998 or 999 lines that have “@” in Element 12, e.g. @main

Controls how the incoming record is loaded

Options are controlled on a “global” level and affect all records loaded


Global Variable Functions

Used to control the following overlay options default record templates used holdings symbol processing field protection automatic dates in records and many more load options...


Commonly Used Triggers

These triggers are found in most load tables:

- @main - @marc - @msg - @recs (not found in Authority load tables) - @atab (not found in Patron load tables)


@main

This trigger identifies the primary record to be created during the load

Will always correspond with Element 11, Pass Indicator

This can only be set to: “b” for Bibliographic records “a” for Authority records “p” for Patron records


@marc

This trigger is used to determine which records can have MARC fields

The settings of this trigger corresponds with Element 9, MARC/Non-MARC Indicator

Typically this is set for Bibliographic, Item, Holdings/Checkin and Authority records (and not Patron records)


@msg

This trigger adds a message to the top of the Output Messages display when a record load starts in Data Exchange

Usually used to specify types of records loaded


@recs

This trigger is used to specify the load table (m2btab file) during interactive loading

For all load tables (except Authority load tables) this trigger equals the m2btab’s file extension

Authority load tables do not have this -- @atab does the same for those tables


@atab

This trigger is used to set the Authority load table to use

In Authority load tables, this should equal the m2btab’s file extension

In Bibliographic load tables, this should equal “a”

This trigger is not found in Patron load tables


Updating/Creating Load Profiles

X > Load Profile Maintenance Naming conventions Options for editing load tables (m2btab) Verifying load tables and checking for

errors Saving changes to load tables Restoring load tables if mistake is made


Practice Exercise 1

“Copying and Editing an Existing Load Table (m2btab)”

Turn to page 111 and complete practice exercise 1


Most Commonly Used Overlay Triggers

@ov_tag @ov_action @ov_protect @ov_rec_number @ldx


@ov_tag

Sets the overlay match point Match point must be an indexed

variable-length field Provisions can be made for specific

matches on field group tag, MARC tag and index tag

Alternate match points can be set


@ov_action

Sets the action to be taken if a match is found or not found

Possible actions are: Insert a new record Overlay an existing record Link records to a Bibliographic record Reject the record


@ov_protect

Specifies fields to protect from overlay Can be set for variable-length (V) and/or

fixed-length fields (F) Variable-length fields can be protected

based on field group tag and MARC tag Variable-length fields have a variety of

protect options (:d and :k)


Overlay Based on Innovative Record Number

@ldx is used to overlay using the Innovative record number as the match point

Options for @ldx include the @ov_rec_number trigger

@ldx overrides the @ov_tag and @ov_action triggers


Overview of Overlay Triggers

MOST USED

@ov_tag @ov_action @ov_protect @ov_rec_number @ldx

LESS USED

@ov_attach_delete @ov_attach_insert @ov_priority @ov_priority_action @ov_title


Overlay using Innovative record number as Match Point

@ov_rec_number

@ldxmatch?

Add record tothe database

Overlay existingrecord

Do not add recordto the database

Use @ov_tag and@ov_action tooverlay record

YesBEGIN ofrecord load

END ofrecord load

1

1

“t”

“r”

No

“i”


Overlay using Indexed Control Number as Match Point

@ov_action1 or 2 matches

Additional matchpoint?

@ov_action0 matches

Add record tothe database

Obey @ov_protect rules and overlayexisting record

Do not add recordto the database

2

2BEGIN of

Record load

Reject

Insert

END ofrecord load

1

1

Yes

@ov_tagmatch?

No No

Reject

Overlay

Yes

Insert


Training Exercise 2

“Practice Applying Overlay Triggers”



Busy Record Handling Triggers

Loading program will not overlay “records in use by system”

Two triggers control the handling of “busy” records: @busy – insert busy records? @busy_file – save busy records to file that

can be loaded later


Automatic Date Selection Triggers

Load system date into fixed-length fields @cdate=“y” - automatic Cat. Date @odate=“y” - automatic Order Date @rdate=“y” - automatic Recv Date


@holdsymb

Innovative staff can construct a Holding Symbol table to issue loading instructions for: Bibliographic Call Number Bibliographic Location code 6XX fields based on 2nd indicator values

(e.g., value of 0 to load LCSH only) Holding symbol processing is activated

using @holdsymb trigger


@dflt

Fixed-length field values are assigned from the default Record Template when no value for the fixed field is present in the incoming record

The format is:@dflt=“biblio,item”(“pass 0, pass 1”)


@link

Linked (aka attached) records are loaded based on settings of @link trigger

The format of this trigger is:@link=rectype:pass#:MARC/count:tag

@link=“i:1:949 1”@link=“i:1:945:b”

@link=“i:1:#1”


Additional Triggers

@comline @diac_sub_table @locmerge @m2b_normalize_020 @password (applicable if your library

has multiple accounting or serials units) @poprint


Practice Exercise 2

“Specifying Record Templates, the Cat. Date Field and Creating Linked

Records”



Special Processing Functions (%)

Specified in 001-997 load lines that have “%” in Element 12, e.g. %008=“y”

MARC tag is processed in a special way when loaded

Examples of usage: 001 and 008 field processing Encrypt patron PINS Foreign currency conversion Activate a translation table (m2bmap)


%001

This function has five parts:%001(bcode,char,skip,start,valid)

bcode - RLIN code to Bib Code 1; set to “n” since RLIN db no longer used

char - valid characters for 001 field skip - number of characters to skip start - valid chars. for start of field valid - 001 have an OCLC prefix?


Commonly Used Special Processing Functions

%008 - places Leader after 008 %bracket - place [ ] around GMD %encryptpin – encrypt patron PIN %foreign – foreign currency processing %replace – single character replacement %map - activates a translation table

(m2bmap) %first and %last – select first or last MARC

tag occurrence


Training Exercise 3

“Using Special Functions”



Command Functions (#)

Specified in 998 or 999 lines that have “#com” in Element 12

For example: #com=“ins”@password=“” Activated using @comline trigger during

OCLC interactive downloading (not batch)


More on Command Functions

Functions to: Specify fixed-length field information Activate triggers from command line Create linked records

Can new command functions be created locally?

See the Innovative Guide and Reference for more information about these commands


Practice Exercise 3

“Troubleshooting Load Tables 101”



Translation Tables (m2bmap)

Used to change the data in incoming record during the load, e.g. change ABC to 123

Translations are stored in an m2bmap file

Activated in m2btab with the special processing function %map

More than one m2bmap file can be used in a load table


Data Elements in Translation Tables<comparison> | <replacement>

Comparison expression is a string or regular expression to search for in incoming data file

If match is found, comparison expression will be replaced by the replacement expression


Data Elements in Translation Tables@special_instruction=

Global instructions appear at top of file and start with @

They affect all lines in the m2bmap file @stop_on_map=true @delimiter=| @case=true


Example of Simple Translation Table

@stop_on_map=true@case=trueJuv Ref|mjrefSouth|sSJUV|sjuvant|manthJUV|mjuv


Using Regular Expressions

Regular Expressions can be used in comparison expressions

These make use of “meta-characters” listed on page 24

More information on regular expressions can be found in any basic UNIX manual


Syntax for Regular Expressions

Comparison expressions(.*)$0|d(.*)$1

Replacement expressions \0|d19\1


Tips on Using Translation Tables Order of lines in a map file is important

longest matching field should appear first Avoid unwanted case changes

@case=true Loading data into fixed-length fields

byte count in Element 5 must be > largest comparison

Must activate in load table Specify m2bmap in Element 12 using

%map trigger


Creating and Updating Translation Tables

X > Load Profile Maintenance E > Edit m2btab and m2bmaps T > TRANSLATION Tables (m2bmap) N > Create a NEW m2bmap file Use %map trigger to activate translation

table in m2btab Must load records to test m2bmap file


Training Exercise 4

“Translation Tables (m2bmap) ”



Data Analysis before Profiling

Issues to consider before Profiling - MARC tags to load - Creating new records or overlay existing records - Remapping of MARC data - Loading linked records


Use Data Exchange to Transferand Prep Files

Launch Millennium Cataloging Go | Data Exchange Select Process: Load records via

locally-created load profiles (.local) Data file must be transferred with “.lfts”

file extension (use “Get FTS” or “Get PC” buttons to transfer files to server)

“Prep” file to create “.lmarc” file for loading


Use Data Exchange toView Records

Click on “.lfts” or “.lmarc” file Click “View” button Viewing features include:

“Count” button (counts number of records and MARC tags)

“Search” button (to find records that match specific criteria)


The m.marcload.local file

What does this file do? Data elements in this file Edit this file to create “Load” buttons for

your new load tables X > Load Profile Maintenance L > Edit m.marcload.local change –fzzz to –f<ext of m2btab.> Test new “Load” buttons in Data

Exchange


Practice Exercise 4

“Applying Required Changes to an Existing Load Table (m2btab)”



Test Load Files Using Data Exchange

Admin | Clear Initials Admin | Reset Initials Go | Data Exchange Select Process: Load records via

locally-created load profiles (.local) Click once on “.lmarc” file Click correct “Load” button Click “Test” button at the bottom of

screen


Practice Exercise 5

“Overlaying of Bibliographic Records”



Review: Creating a New Load Table (m2btab)

Plan load table settings View MARC records to be loaded Copy an existing load table Verify and check load table for errors Restore load table if errors are made


Review: Using Translation Tables (m2bmap)

m2bmaps are optional But if you need one, remember to

activate the m2bmap in the m2btab file Specify m2bmap in Element 12 using

%map trigger


Review: Editing m.marcload.local

Add m2btab file extension or brief description to Element 1 to distinguish between “Load” buttons

For example:C > LOAD a MARC file (.pbursar)

Use Clear Initials and Reset Initials to activate new “Load” buttons in Data Exchange


Support from Innovative

Consultation [email protected] csdirect.iii.com

Technical Assistance Profiling Services

Product Code 830 [email protected]


Review of Training Goals

Understand the database structure Understand existing load tables (m2btab) Be familiar with issues to be considered for

new load profiles Update and create load tables Create simple translation tables (m2bmap) Add “Load” buttons to Data Exchange Test load profiles before production


Trainer Evaluation

Workshop Evaluation Form is available on csdirect.iii.com

Select “Contact Us” in upper right corner "Feedback" from left column "Workshop Feedback" from right column

Please take a few minutes to fill out the evaluation. Thank you!


Load Profile Training. Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Agenda Day.

Documents