Top Banner
Voyage meets MeLCat: MC’ing the Introductions
47

Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Jan 19, 2016

Download

Documents

Stewart Chapman
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Voyage meets MeLCat:MC’ing the Introductions

Page 2: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

MeLCat extract sequences

Voyager

bibout.pl

bib extract

patout.pl

today’s extract

yesterday’s extract

patdiff.pl

Pmarcexport

patron extractto MeLCat

Page 3: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Our MeLCat implementation is unfortunately still a work in progress.

The information presented here may be subject to change.

The information is as current as possible, given deadline constraints.

Disclaimer

Page 4: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

The bib extract and patron extract programs, and the patron differences program, are all written in Perl and are designed to run on your Voyager box.

A Unix box is assumed. Minimal modifications should be required in order to run on a Windows box.

Notes

Page 5: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

User requirements:

Familiarity with a text editor, file names, and paths on your machine.

You’ll need to be able to query your database (SQL provided).

You will be editing the setup section at the beginning of the programs to customize settings for your institution.

Notes

Page 6: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

MeLCat extract sequences

Voyager

bibout.pl

bib extract

patout.pl

today’s extract

yesterday’s extract

patdiff.pl

Pmarcexport

patron extractto MeLCat

Page 7: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

/m1/voyager/YOURdb/sbin/Pmarcexport \

-oOUTPUTFILE \

-mBtoday-N \

-ttoday-N \

-i

Bib Extract, Pmarcexport

The export command is in script form in a file. Thus the parts of the command have been broken out. The trailing “\” indicates the command continues on the next line.

Page 8: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

/m1/voyager/YOURdb/sbin/Pmarcexport \

-oOUTPUTFILE \

-mBtoday-N \

-ttoday-N \

-i

Bib Extract, Pmarcexport

Substitute your own values for words in upper case.

Page 9: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

/m1/voyager/YOURdb/sbin/Pmarcexport \

-oOUTPUTFILE \

-mBtoday-N \

-ttoday-N \

-i

Bib Extract, Pmarcexport

The mode, m, is B, which means to select records by both create and update dates.

The export target, t, gives the date range.

The last parameter, i, says to ignore suppressed bib records.

Page 10: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

/m1/voyager/YOURdb/sbin/Pmarcexport \

-oOUTPUTFILE \

-mBtoday-N \

-ttoday-N \

-i

Bib Extract, Pmarcexport

today-N specifies to look back N days from today

At WMU, we used a value of 10,000 for our initial load. For ongoing loads, we use a value of 1, since we run our MeLCat bib extract daily.

Page 11: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract, Pmarcexport problems

We had problems with Pmarcexport.

Our initial large bib export resulted in two files, not one!

This turned out to be expected, but previously undocumented, behavior.

Page 12: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract, Pmarcexport problems

We had problems with Pmarcexport.

Our initial large bib export resulted in two files, not one!

This turned out to be expected, but previously undocumented, behavior.

Had problems with record counts when a split occurred.

Page 13: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract, Pmarcexport problems

We had problems with Pmarcexport.

Our initial large bib export resulted in two files, not one!

This turned out to be expected, but previously undocumented, behavior.

Had problems with record counts when a split occurred.

Also had mysterious SQL error which didn’t(?) seem to affect the output.

When we finalized this presentation, these issues had not yet all been resolved.

Page 14: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

MeLCat extract sequences

Voyager

bibout.pl

bib extract

patout.pl

today’s extract

yesterday’s extract

patdiff.pl

Pmarcexport

patron extractto MeLCat

Page 15: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

bibout.pl voy_bib_export_file melcat_bib_file

voy_bib_export_file is created by Voyager’s Pmarcexport program

melcat_bib_file is the resulting bib file ready to be sent to MeLCat

Both files are in MARC format.

Bib Extract, Usage

Page 16: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Typically, Pmarcexport will create one output file. For large amounts of data, you may have two or more files.

In that case, you can run bibout on each file, or possibly concatenate the Pmarcexport files into one for bibout to process.

In this presentation, and in bibout.pl, Pmarcexport’s output is referred to as one file.

Bib Extract, Usage

Page 17: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Two stages:

test extracts and initial loads

ongoing production extracts and loads

The exact usage is determined via a Pmarcexport script parameter and the corresponding setting for bibout.

Bib Extract, Usage

Page 18: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract – What does it do?

bibout reads the file created by Pmarcexport.

For each bib, find and scan its mfhd(s) for fields 866, 867, and 868. These fields, if present, are added as is to the bib record.

Get mfhd and item information associated with this bib record, or just mfhd data as a limited fallback when there is no item record.

Page 19: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract – What does it do?

Append a new field to the bib record that contains the following data:

|a item id (mfhd id if no item) |b item barcode |c item status |d item type |e call number |f location |g enum info |h number of recalls

This is field 999 by default.

Page 20: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract – What does it do?

|a item id

When there is no item id, |a contains a mfhd id. This is indicated with a leading “M” in this field.

Still to come: handling deleted records

Page 21: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract, database access

bibout needs to know how to connect to your Voyager database.

At WMU, we’ve standardized on storing this information in a file, allowing for easier updates and better security. Any programs that access Voyager do not have the specific database access information in the program.

Page 22: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract, database access

The information you need for this is your:

machine name

database name (may be username)

database login (password)

Oracle SID [VGER | LIBR]

Voyager 5’s SID is VGER; the SID for previous versions is LIBR. Use the right one for your version.

Page 23: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract, database access

Put this information in the file, all on one line:

your.machine.name.edu/namedb/db_login/sid

(namedb may be username in your situation)

The slashes, “/”, belong in there, and the bib extract program will read this data so it can access Voyager. Make sure that this file contains your information, including the SID.

Then we need to tell bibout where this file is, by specifying its path and the file name.

Page 24: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract - which database?

bibout needs to know from which database we are extracting the data.

Since the database name may not equate to the user, we specify the database name separately.

For example, you may be connecting to your database as a read-only user, so you would have to specify the database.

Page 25: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract

bibout also needs to know how many days to look back.

Give it the same value, N days, that you used in the export script.

This makes sure that everyone is on the same page.

Page 26: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract – Which locations?

We wanted to restrict, by location, the items showing up in MeLCat.

Here’s a sample query to see your location codes:

Select the location_codes you want, and put them in a text file, one per line.

Page 27: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract – Which locations?

After creating the location codes file, you need to tell bibout where this file is (the path), and the file name.

Page 28: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Bib Extract

By default, the new field that contains the item/mfhd information is 999.

If you need a different tag, you’ll need to tell bibout what that is.

Page 29: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

MeLCat extract sequences

Voyager

bibout.pl

bib extract

patout.pl

today’s extract

yesterday’s extract

patdiff.pl

Pmarcexport

patron extractto MeLCat

Page 30: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Extract, Usage

patout.pl patron_extract_file

patron_extract_file is the double quoted, comma separated value format patron data retrieved from Voyager

Page 31: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Extract – What does it do?

Similar to bibout, the patron extract program (patout.pl) also looks back so many days to get patron data.

Voyager is queried and the patron extract data for MeLCat is put in CSV format.

Page 32: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Extract, Output

last name, first name, middle name,

patron barcode, patron group, patron exp date,

WIN (WMU ID number), patron ID (Voyager internal number),

delete indicator (D=delete, blank to retain), blocked (Y/N),

email address, email exp date

patron exp date is 2382.12.31, if indefinite

Page 33: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Extract, blocking

Patron is blocked when:

patron record expired before today

lost items count exceeds lost limit defined in the extract

recalls count exceeds recall limit defined in the extract

fees due amount exceeds the maximum balance for the patron group used in the extract

email address record expired before today

Page 34: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Extract, assumptions

The extract uses the patron’s most recent active patron group as the patron group of record.

Page 35: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Extract, database access

The patron extract gets its Voyager database access information from the same file mentioned earlier in the bib extract part of this presentation…

Database Access

Put this information in the file, all on one line:

your.machine.name.edu/namedb/db_login/sid

(namedb may be username in your situation)

The slashes, “/”, belong in there, and the bib extract program will read this data so it can access Voyager.

…then we need to tell patout where this file is, by specifying its path and the file name.

Page 36: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Extract

Again, like with bibout, we need to tell patout how many days back to look to get patron data.

This value will probably depend on how often your patron data is updated.

Page 37: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Extract

We discovered setting up limits by patron group to be pretty much impossible. This is because a patron group can be in multiple circ groups, with each circ group able to have different limits.

So we went with across-the-board limits, and you’ll have to use a global value for the lost-limit and the recall-limit.

Page 38: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Extract, patron groups

There will probably be some patron groups that should not be in MeLCat.

You’ll need to tell patout which patron groups are allowed. This is done via the patron group ID.

Here’s how to get these ID numbers:

Page 39: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Extract, patron groups

Now you have a listing of patron group IDs and names. Note that fine/fee limits can be found in the Systems Administration client by clicking on Circulation, then on Policy Definitions.

Record your patron group limits at each corresponding patron group ID.

Remember that in Voyager, money values (U.S.) are stored in cents, so the values you enter here, divided by 100, equal the dollar amounts.

Page 40: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Extract

We also need a representative circ group for patout to do its thing.

From your Sys…Admin… client, pick a circulation policy group with the most appropriate lending attributes. Now run a query to get its ID number:

and let patout know what it is.

Page 41: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Extract

Finally, patout needs to know where to write the output. Specify the file path and file name.

The filename will automatically have a date stamp appended.

For example, if you’re using a file name of “patron.out”, you’ll end up with file names like patron.out.20060825

patron.out.20060826

etc.

Page 42: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

MeLCat extract sequences

Voyager

bibout.pl

bib extract

patout.pl

today’s extract

yesterday’s extract

patdiff.pl

Pmarcexport

patron extractto MeLCat

Page 43: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Differences Extract

patdiff.pl looks at the current and the previous patron extract and puts the differences into the patron extract to be sent to MeLCat.

The extract comparison is based on patron ID.

Page 44: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Differences Extract

The extract gets:

new records

records no longer showing up; they aremarked for deletion

the current version of any records with differences in them

Page 45: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Patron Differences Extract

patdiff needs to know where to write its output file. Specify the file path and the file name.

Page 46: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Extracts ready!

Your bib and patron extracts are now ready to be sent to MeLCat.

Page 47: Voyage meets MeLCat: MC’ing the Introductions. MeLCat extract sequences Voyager bibout.pl bib extract patout.pl today’s extract yesterday’s extract patdiff.pl.

Thanks for listening.

Questions?

[email protected]

269.387.3885

Picture © 2005 by Roy Zimmer