Vista Plus Technical Addendum

Vista Plus

Technical Addendum

This document discusses some technical aspects of certain Vista Plus® functions. This information is presented in an Addendum due to its technical nature and because it is probably of interest to only a small percentage of Vista Plus users.

VP050502-TA-EN-02

Vista Plus

Technical Addendum

VP050502-TA-EN-02

Rev: 2013-Feb-14

This documentation has been created for software version 5.5.2. It is also valid for subsequent software versions as long as no new document version is shipped with the product or is published at https://knowledge.opentext.com.

Open Text SA

40 Avenue Monterey, Luxembourg, Luxembourg L-2163Tel: 35 2 264566 1

Open Text Corporation

275 Frank Tompa Drive, Waterloo, Ontario, Canada, N2L 0A1Tel: +1-519-888-7111Toll Free Canada/USA: 1-800-499-6544 International: +800-4996-5440Fax: +1-519-888-0677

Email: [email protected]

FTP: ftp://ftp.opentext.com

For more information, visit http://www.opentext.com

Copyright ©2006 ‐ 2012 Open Text Corporation

OpenText is a trademark or registered trademark of Open Text SA and/or Open Text ULC. The list of trademarks is not exhaustive of other trademarks, registered trademarks, product names, company names, brands and service names mentioned herein are property of Open Text SA or other respective owners.

Disclaimer

No Warranties and Limitation of Liability

Every effort has been made to ensure the accuracy of the features and techniques presented in this publication. However, Open Text Corporation and its affiliates accept no responsibility and offer no warranty whether expressed or implied, for the accuracy of this publication.

Vista Plus Technical Addendum i

Table of Contents

Introduction ............................................................................................................................. 1ImportFileInfo File Format ....................................................................................................... 2

Passwords and ImportFileInfo.......................................................................................... 4Exporting LDAP User and Group Data.................................................................................. 5

What DSExporter Copies.................................................................................................. 5What DSExporter Does Not Copy ................................................................................... 5Output File Format............................................................................................................ 6DSExporter Command Format........................................................................................ 6After Running DSExporter................................................................................................. 7

Using the –x Option During Report Capture ........................................................................ 8–x Prepend File Location.................................................................................................. 8Sample Uses and Files ...................................................................................................... 9Notes ................................................................................................................................ 10

Vista Plus Encryption ............................................................................................................. 11Vista Plus Temporary File Use ............................................................................................... 12

The Warehouse Parameters Temporary Directory...................................................... 12The TMP/TMPDIR Temporary Directory ......................................................................... 13The .locks Subdirectory .................................................................................................. 13

Report Warehouse Folder and File Structure ..................................................................... 14The server.debug.log Message File .................................................................................... 17

Circular Logging ............................................................................................................. 17External Password Authorization ......................................................................................... 19

About the Sample Authorization Module.................................................................... 20Building the Sample Code ............................................................................................ 21Installing a Plug-in Security Module .............................................................................. 22Using the Sample Module ............................................................................................. 22Making a Custom Authorization Module..................................................................... 22

The Migration Process........................................................................................................... 24The Migration Process .................................................................................................... 24Offline Volume Structure................................................................................................ 25VMTransport’s Temporary Files ...................................................................................... 26

Font Support in Web View When Using Xvfb ..................................................................... 28Using Vista Plus Utilities .......................................................................................................... 29

The check_gens Utility.................................................................................................... 29The del_gens Utility ......................................................................................................... 31

Adding Index Values to the Database .............................................................................. 34Space Considerations.................................................................................................... 35convert_gens Command Format................................................................................. 36

Table of Contents

ii Vista Plus Technical Addendum

Removing Index Values from the Database ..................................................................... 38revert_gens Command Format .................................................................................... 38Using revert_gens............................................................................................................ 39Reclaiming the Database Space................................................................................. 40

Tuning Vista Plus Server Configuration................................................................................ 42Scalability Configuration Options................................................................................. 42Recommended Kernel Parameters on the Server Host ............................................. 46MySQL Memory Limits..................................................................................................... 51Preventing Lost Oracle Database Connections......................................................... 51

Vista Plus Technical Addendum 1

IntroductionThis document discusses some technical aspects of certain Vista Plus® functions. It covers these topics:

• File format for importing user, group, and link information with the vadmin ImportFileInfo command

• Using the DSExporter command to export user data from an Active Directory or LDAP database so it can be imported in to Vista Plus

• Using the capture –x option to prepend PCL files to captured reports

• How Vista Plus encryption works

• Temporary files and directories Vista Plus creates and uses

• Report warehouse folder and file structure, including rendition files created by epurposing™

• The server.debug.log message file

• Creating and using an external password authorization routine

• Details of the VMIdentify and VMTransport processes

• Notes about font support when using Web View in the Xvfb environment

• Using the check_gens and del_gens utilities

• Using convert_gens to add index values from the report warehouse to the Vista Plus database

• Using revert_gens to remove index values from the Vista Plus database if desired

• Topics related to tuning Vista Plus configuration and related settings to improve performance and avoid certain types of errors. These include:

• Scalability configuration options

• Kernel parameter settings on Sun® Solaris® and HP-UX

• Adjusting MySQL memory limits

• Preventing lost connections to an Oracle database

This information is presented in an addendum rather than in the Vista Plus Server Administration Guide due to its technical nature and because it is probably of interest to only a small percentage of Vista Plus users.

ImportFileInfo File Format

2 Vista Plus Technical Addendum

ImportFileInfo File FormatThe ImportFileInfo vadmin command loads multiple users, groups, and/or user-group links into Vista Plus from a text file. It can also modify or delete existing users, groups, and links. As described in the Vista Plus Server Administration Guide, the command format is simple: the only required parameter for ImportFileInfo is the name of the text file containing the information to import. Here is the syntax:

vadmin c=ImportFileInfo f=file [ptype=type] [password=password]

Based on the information in the text file, ImportFileInfo builds a batch file of the appropriate vadmin commands, then submits it. The optional ptype and password parameters determine whether users created by ImportFileInfo use passwords found in the text file or are assigned different passwords. See the section “Passwords and ImportFileInfo,” below.

Tip The vadmin executable must exist in the current working directory where you enter the ImportFileInfo command. This is true even if the vadmin installation directory is included in your PATH statement.

The key to using ImportFileInfo successfully is the format of the data in the text file. A single text file can contain all three types of records to update—users, groups, and user membership in groups—and all three actions—adding, modifying, and deleting. The file format is as follows:

• Each record occupies one line.

• Each line consists of a number of fields; fields are delimited by the vertical bar character ( | ). The number of fields depends on the type of record—user, group, or link—being updated.

• The first field of the line determines the type of record to update:

• 0 = user

• 1 = group

• 2 = user-group link

• The second field of the line determines the type of action to take:

• 0 = add new record

• 1 = modify existing record

• 2 = delete existing record

• The rest of the line contains the information to add, modify, or delete.



• A line for a user record has this format:

0|action|login-name|full-name|description|password|home-folder| unix-uid|admin (y|n)*|primary-group|fax-number|pager-number| email-address|ptrdef|home-address|department|start date|end date| days till password expires|authorization method (v|u|x|a|n)*|force password change? (y|n)*|

*: Allowable values for the field are shown in parentheses; enter only the value.

• If you include ptype on the command line, any passwords in the file are ignored. See “Passwords and ImportFileInfo” on page 4.

• A line for a group record has this format:

1|action|group-name|description|home-folder|ptrdef|start date|end date|

• A line for a user-group link has this format:

2|action|user-name|group-name|start date|end date|

• Blank spaces in data fields are significant; they are not ignored. Do not include a space at the beginning or end of a field unless you want that space included when the data is added to Vista Plus. For example, entering a user name as | Chris| is not the same as |Chris|.

• Not all data is required; required fields for each action are the same as for the corresponding vadmin command. For example, a line which is adding a group must start with 1|0 and include a group name and home folder; the other fields are optional (but see the next item). See the Vista Plus Server Administration Guide for descriptions of each field and which fields are required.

• If you leave one or more fields out, you must include the correct number of vertical bars as placeholders, even if the omitted fields are at the end of a line. There must be one | character for each field, including blank fields. For example, to add the group Writers with a Home folder of Marketing and a start date of 12/01/05, but no description, default printer definition, or end date:

1|0|Writers||Marketing||12/01/05||

The two vertical bars between Writers and Marketing cause Marketing to be treated properly as the Home folder rather than the description; the two vertical bars after Marketing indicate the default printer definition is left blank, and the two vertical bars at the end indicate the end date is blank. A group line contains eight fields, including the 1 which marks it as a group record; the sample line includes eight | characters, as it must.

• The order of the records in the file does not matter. The groups are added/deleted/modified first, then the user records, then the user-group links.

• You can include comment lines in the file. Start each comment line with a pound sign (#).

• You can include blank lines in the file; they are ignored.



Notes• If a user or group record specifies a Home folder which doesn’t exist, the folder is

created as a subfolder of the MAIN folder.

• All groups included in either user or link records must either already exist or must be created by a group record line in the same file. A group can be added only with a group (type 1) line.

• If you attempt to perform an add (action type 0) and the record already exists, no action is performed. The existing record is not updated with new information. You must use an update (action type 1) to update an existing record.

Passwords and ImportFileInfoDepending on the source of the data for ImportFileInfo, your text file may or may not contain user passwords. If there are passwords in the source text file and you want to use them in Vista Plus, do not include the ptype parameter. If the source file doesn’t have passwords, or if you want to assign users new passwords during the import, you can use ptype, and optionally password. There are three possible values for ptype:

• common: Gives all users the password you specify using the password parameter. You must include password with this setting.

• random: If used without password, sets each password to the user name followed by a five-digit random number. If you include password, sets each password to the password entry followed by a five-digit random number. (If you set password to null—password=””—the password is just the random number.) When you use random, vadmin creates a text file called random.out which lists the user names and the randomly-generated passwords.

• username: Sets each password to the user name.

For example, to set all passwords to the word new followed by a five-digit random number:

vadmin c=ifi file=file.txt ptype=random password=new

When you use random, users will not be able to log in to Vista Plus until an Administrator tells them what their new passwords are, since each user will have a different random number at the end of his or her password. Using common or username will let users log in, but does not provide much security.

If you do not include ptype, ImportFileInfo uses any passwords found in the text file. If there are no passwords in the text file, the users are created without passwords.

Including password without ptype has no effect; ImportFileInfo still uses passwords from the text file, if there are any.

Note Important! The ptype and password options are applied both to new users being added and to existing users being modified by ImportFileInfo, and override any passwords contained in the source text file. Be careful not to inadvertently change passwords for existing users.

Exporting LDAP User and Group Data


Exporting LDAP User and Group DataThe Vista Plus DSExporter command takes user and group data from a Microsoft Active Directory or other LDAP user database and exports it to a text file. You can then—after some modifications—use the text file as the source file for the vadmin ImportFileInfo command to create the groups and users in Vista Plus.

DSEXporter is frequently used together with the Vista Plus LDAP Authentication module to create Vista Plus users for LDAP users, who can then sign in to Vista Plus using their LDAP names and passwords. The LDAP Authentication module is described in the Vista Plus Server Administration Guide.

Note DSExporter is included in a full installation of the Vista Plus Windows server. To install just DSExporter, perform a custom installation of the server and select only the DSExporter option. See the Vista Plus Server Installation Guide chapter on Windows server installation for more information.

What DSExporter CopiesDSExporter takes all of the information stored in LDAP for each group and user and copies it to the output file. For groups, this is only the group name. For users, it includes:

• User name

• Full name

• Description

• E-mail address

• Group memberships

If a user belongs to only one group, that group becomes his or her primary group when imported into Vista Plus. If a user belongs to more than one group, all groups are added as secondary groups; the user will not have a primary group when imported into Vista Plus unless you take further action, as described below.

What DSExporter Does Not CopyThe information listed in the previous section is all that is available to DSExporter. It does not include enough information to create valid Vista Plus user and group records:

• All Vista Plus users must have either a home folder or a primary group. There are no home folders in LDAP, and if a user belongs to multiple LDAP groups, DSExporter has no way of knowing which should be the primary group in Vista Plus.

• All Vista Plus groups must have home folders. There are no folders for groups in LDAP.

Because this information does not exist in LDAP, but is required in Vista Plus, DSExporter includes features to add a default primary group for each user and a home folder for each group to the created text file, as described in the next section. This ensures that the text file



contains at least the minimum information required to create valid users and groups in Vista Plus.

Additionally, DSExporter cannot copy user passwords from LDAP, as they are encrypted. If desired, you can use an ImportFileInfo option to set passwords for each user during import. However, if the Vista Plus users will be using either operating system passwords or their LDAP passwords (through the LDAP Authentication module) to log in to Vista Plus, there is no need to create Vista Plus passwords for them.

Note You can also type each user’s password into the text file before importing the information into Vista Plus, but this is rarely practical, and would be a security risk as the passwords would have to be entered in plain text.

Output File FormatDSExporter creates a text file containing the exported groups, users, and user-group links. The file is formatted for input into Vista Plus using the vadmin ImportFileInfo command. As not all data accepted by Vista Plus is available, many of the fields on each line are blank. DSExporter places the correct number of vertical bars on each line so that it will import into Vista Plus correctly. For more information, see “ImportFileInfo File Format” on page 2.

The default output file name is DSExporter.txt in the current directory. You can change the file name and location using the x option.

DSExporter Command FormatTo export LDAP data to the intermediate text file, simply run DSExporter. You specify the host or domain to copy data from. Options let you set the output file name, default primary group and Home folder, and more.

The format for DSExporter is:

DSExporter s=source [t=w] [options]

Source is the name or IP address of an LDAP server, or a Microsoft Exchange or Active Directory domain.

You must include t=w if the source is anything other than a Microsoft Exchange server.



The available options are:

After Running DSExporterAfter you have run DSExporter, you can use its output file as the input file for vadmin ImportFileInfo. However, you may want or need to make any or all of these changes to the data in the output file before using it:

• Setting the correct home folder for each group being created.

• Setting the correct home folder for users who should have one.

• Setting the correct primary group for users who were in more than one LDAP group.

• Adding any other available data. It may be easier to add this data to the text file before import rather than editing the groups or users in Vista Plus.

For a description of how to use vadmin ImportFileInfo and the information you can import with it, see “ImportFileInfo File Format” on page 2.

Option Description

a=v|u|x|a|n Sets the authorization method for all users in the output file:

• v = Vista Plus passwords. This is the default.

• u = User-provided authorization. Use this method if you want to use the LDAP authentication module on a UNIX server host.

• x = Unix passwords.

• n = Windows passwords. Use this method if you want to use LDAP passwords on a Windows server host.

• a = Any authorization method.

Authorization methods are described in the Vista Plus Server Administration Guide.

c=y|n Setting to y forces users to change password when first logging in to Vista Plus. Default is n. Has an effect only if authorization method is v or a.

f=folder Sets the home folder for all groups in the output file to folder. If you don't use this option, all Home folders are set to HOME_FOLDER.

g=group Sets the default primary group for users in the output file to group. This is used for all users except those who belong to exactly one group in the LDAP database. If you leave out this option, the default primary group is PRIMARY_GROUP.

x=file Sets the output file. You can enter a full path or just a file name. The default output file is DSExporter.txt in the directory where you run DSExporter. If the file already exists, the information is appended to it; if it doesn't exist, it is created. If you include a path, the directory must already exist.

Using the –x Option During Report Capture


Using the –x Option During Report CaptureThe capture –x option lets you prepend a file to a report during capture. You can use it either on the capture command line or by including it in a capture configuration file. The Vista Plus Server Administration Guide describes the syntax for using –x; however, it does not discuss what you may want to use this option for or the proper format for prepend files.

–x should be used only with ASCII text reports and some PCL reports. While –x can add any file before a captured report, it should be used only to prepend files containing PCL codes. Its most common use is to emulate Printer-Specific Environment Settings usually used by the printer a report is printed to. If a printer uses specific mode (portrait or landscape) or font settings, reports sent to it don’t need to include these settings—they use the mode and font preset for the printer. When one of these reports is captured to Vista Plus, it may not display or print correctly because the report data does not include the mode and font settings it needs.

For example, a printer could be set to always print in landscape mode using compressed print. All reports sent to this printer use these settings; the reports themselves do not include any commands setting the mode or compression. When one of these reports is captured to Vista Plus and displayed, it will display in portrait mode using normal print: its data may not all fit on the page, characters could overprint at the ends of lines, or there could be other problems.

The –x option can solve this problem—and various others—by prepending to the report a file containing a set of PCL codes. These PCL codes can replicate the Printer-Specific Environment Settings so the report displays correctly in Vista Plus clients and prints correctly when printed from Vista Plus.

Note If you prepend a file containing PCL codes to a captured text report, the captured report file is changed to a PCL file.

–x Prepend File LocationWhen you use –x, you specify the name of the file to prepend to the captured report. You include just the file name, not a path. The file must exist in the prefiles subdirectory of the Vista Plus installation directory on the server you are capturing the report to. If you use –x from a remote host, the prepend file must exist on the host being captured to, not on the remote host where you are performing the capture.



Sample Uses and FilesAs mentioned above, the reason to use –x is to add PCL control codes to the beginning of captured reports. Here are two sample situations where –x could be used.

UNIX Text FilesUnless otherwise specified, a PCL report expects DOS-style line termination: each line ends with a carriage return/line feed pair. UNIX text files have only a line feed, not a carriage return, at the end of each line. You can prepend a file to UNIX text files so line termination will be correctly recognized. Here are the contents of the file, shown in both hex and PCL representation. The file should be a continuous line of data ending with a newline character.

If you created this file and saved it in the prefiles subdirectory as, for example, unixterm, you could prepend it to any captured UNIX text file to allow it to print properly on a PCL printer.

Note The NORMALIZE flag in the server.cfg file can also correct problems with remote printing UNIX text files. See the Vista Plus Server Administration Guide. This flag does not affect local printing or displaying reports with a Vista Plus client.

Setting Landscape Mode and Compressed PrintPrepending the following sample file causes a report to print in landscape mode using compressed print. You could use it in the situation described earlier, where the report is usually printed to a printer which uses these Printer-Specific Environment Settings. While it is shown in three lines here, this should be a continuous string of data ending with a newline character.

Hex Representation: 1B 45 1B 26 6B 32 47

PCL Representation: Esc E Esc & k 2 G

Hex Representation: 1B 45 1B 28 38 55 1B 28 73 34 30 39 39 54

PCL Representation: Esc E Esc ( 8 U Esc ( s 4 0 9 9 T

Hex Representation: 1B 26 6C 31 4F 1B 26 6C 38 44 1B 26 6C 35 45

PCL Representation: Esc & l 1 O Esc & l 8 D Esc & l 5 E

Hex Representation: 1B 26 6C 36 36 46 1B 28 73 31 36 2E 35 30 48

PCL Representation: Esc & l 6 6 F Esc ( s 1 6 . 5 0 H



Notes• The first character in any PCL prepend file should be the PCL reset sequence:

<Esc>E.

• You can use a binary editor to create files containing PCL codes. You enter the hexadecimal representation of each PCL code.

• The –x option may not work on all PCL files. Some PCL files, particularly those generated by Windows applications, contain PJL header lines. These PJL header lines will prevent the prepended PCL file from having any effect.

• Vista Plus attempts to remove any initial CR/LF characters and the PCL reset sequence from a report when a prepend file is being used. If a reset character follows non-CR/LF characters in a PCL report, the prepended PCL file will most likely be ignored.

• The original file captured into Vista Plus is never modified by the capture process; however, the prepend file is added to the copy of the original file kept in the Vista Plus warehouse (the .v file). This allows you to remote print the report using the PCL attributes in the prepend file.

• If the prepend file you include with –x doesn’t exist, an error message is logged in the Vista Plus server log and returned to the rcapture command or other capture tool being used.

Vista Plus Encryption


Vista Plus EncryptionVista Plus 4.1 and later includes server.cfg flags to enable encryption of information as it passes between the various Vista Plus clients and the Vista Plus server. There are five separate encryption flags, as listed in Appendix B of the Vista Plus Server Administration Guide. Each flag controls encryption of information as it is passed between the Vista Plus server and a particular client: vadmin, the Server Administration client, the Windows Client, Web View, and capture clients.

These flags encrypt only network data packets being sent across the network or the Internet. Vista Plus does not encrypt information stored in the report warehouse, even if all encryption flags are set. When an encryption flag is on, the information is encrypted by the server or client before transmission, and decrypted before being stored or displayed at the other end.

Note A user’s name and password are always encrypted when they are sent to the server during login. This is not affected by the encryption flags.

Note The encryption flag for Web View affects information only when it is being sent between the Vista Plus server and the Web server where Web View is installed. Whether or not data is encrypted between the Web server and the user’s workstation is controlled by the Web server, not by Vista Plus.

Vista Plus Temporary File Use


Vista Plus Temporary File UseVista Plus creates temporary files and/or subdirectories in three directories on the Vista Plus server:

• The temporary directory set during installation and stored in warehouse parameters.

• The directory named by the TMP (Windows) or TMPDIR (UNIX) statement in the server.cfg file. If server.cfg does not have this statement, Vista Plus uses the directory set by the equivalent system environment variable—TMP or TMPDIR.

• The .locks subdirectory of the Vista Plus server installation directory.

The tables below list the files created in each directory.

File name notes for the tables:

• pid refers to the process ID on a UNIX server or the thread ID on a Windows server.

• nnn indicates a numeric file name from 1 – 65535. These file names are assigned sequentially by the operating system, not by Vista Plus.

• GenID and RepID are the generation ID and the report ID, respectively.

The Warehouse Parameters Temporary Directory

File Name Process Description

temp_pid Bundle distribution and remote printing

Used during token replacement for bundle banner pages.

bundle_print.pid Bundle printing Holds printable report data for bundle.

nnn Capture Store portions of the list of objects in the generation when the list is too big for memory.

OLTGenID\GenGenIDOLTGenID\RepRepID

Client-requested archiving

Temporary storage of generation data to be archived. The GenGenID file contains the generation data in Vista Plus format; RepRepID is a flat file containing the data.

Report file name Conversion from 2.x (the vconvert program)

Created for each generation during conversion from 2.5, then deleted

_VISTA_seconds.dat dircapture Holds copy of file while time stamp is being changed; seconds is number of seconds since midnight.

vista_tempRepID_pid Extracting a subreport during search

Contains matching lines or pages. Used as the source if the subreport is saved with Store Report.

nnn Index creation Used during indexing.

lm_ip.out, lm_sem.out Lock manager initialization

Used during lock manager initialization.

Vista Plus Temporary File Use


The TMP/TMPDIR Temporary Directory

The .locks Subdirectory

report.000 Printing Actual print file for report. The extension may be 000, 001, etc.

vista_print.pid Printing Holds printable report data.

vista_mail.pid Printing; sending e-mail attachment

Holds the user-accessible data; removed when action is complete.

tempRepID.pid Search Holds PCL version of extracted matching pages or lines; used only with PCL reports.

nnn Search One or more temp files used during search.

_VISTA_time.cfg Updating capture configuration file

Original configuration file is deleted; temporary file is saved as configuration file.

vistaportsessionnnn Session management Directory to hold persistent data for open reports for session nnn. Port is the port number of the Vista Plus server.


VVSRnnn vadmin gsr Holds data during activity report generation.

SCHnnn Repository search Holds temporary search data

vmt_n\GenID (n is assigned by the operating system)

VMTransport Subdirectories to temporarily hold all generation data (the generation itself, indexes, etc.) during VMTransport. See page 24 for more information.

vmt_n\VolID (n is assigned by the operating system)

VMTransport Subdirectory to hold compressed generations waiting to be copied to an offline tape volume during VMTransport. See page 24 for more information.


GenID.act Any process that opens a generation file—Web View, VMTransport, and so on

Contains a count indicating how many processes currently have the GenID generation open. When the last process closes the generation, the file is deleted; it is recreated when the generation is opened again.


Report Warehouse Folder and File Structure


Report Warehouse Folder and File StructureWhen you create an online volume for report storage, you specify the root path for Vista Plus to use for the volume. Beneath this root path, Vista Plus creates a structured directory tree containing the files for the report generations captured to that volume. The directory tree has this structure:

• Directly beneath the root path, there is a subdirectory called whnnnn. nnnn is the port number for the warehouse. For example, if the warehouse uses the default port of 7980, the subdirectory is wh7980.

• Beneath this subdirectory is a tree based on the sequence numbers of the captured generations. There is a subdirectory for each digit of the sequence number; the files for a particular generation are stored in a subdirectory contained in the subdirectory corresponding to the last digit of its sequence number. This subdirectory which actually contains the generation files is named Gnnn; nnn is the sequence number.

For example, if the warehouse port number is 7980, the files for generation 365 are in the directory Wh7980/3/6/5/G365. The structure is shown in the picture to the right.

Since not all sequence numbers have the same number of digits, any subdirectory at any level can have both the 0, 1, 2, etc. subdirectories for the next digit of a sequence number, and a Gnnn subdirectory containing generation files. In the picture, if there is a generation with a sequence number of 36, the Wh7980/3/6 directory will have a G36 subdirectory in addition to the 0 through 9 subdirectories shown. Also, if there are generations with more than three digits, as in most Vista Plus warehouses, there will be additional levels of subdirectories.



• The Gnnn subdirectory contains all the files for the report generation. This directory will contain some combination of these files (* in the file name is the sequence number of the generation):

• If you use epurposing, the Gnnn directory can contain these additional files and/or subdirectories (* in the file name is the sequence number of the generation):

File Description Notes

*.v Original format report file

Exists only if the warehouse parameter to store the original format file is set for this report type.

*.i### Index files Indexes are numbered sequentially throughout the warehouse. All generations of a report have the same index file names; each one contains the index entries for just that generation.

*.p### Page security Numbered sequentially throughout the warehouse. Used only for page securities created before Vista Plus 5.5.1.

*.p Page security Contains page securities for the generation. Used for all page securities created with Vista Plus 5.5.1 or later.

*.pidx Page security index. Index used to find individual page securities in the *.p file. Used for all page securities created with Vista Plus 5.5.1 or later.

Note When you re-index a generation, its page securities are re-created based on the new index values. If you re-index a report or generation using Vista Plus 5.5.1 or later, its page securities are moved from *.p### files to the *.p and *.pidx files for the generation. (If you only re-index one index, only pages securities that use that index are moved.)

*.o Object file Describes the contents of the generation: where each text object goes; fonts to use; where bitmap images go, and so on.

*.t Text file Contains the text objects for the report, with no formatting or location information. All formatting and location instructions are contained in the *.o file

*.b Bitmaps Contains all bitmap images for the generation; the *.o file contains instructions for placing the bitmaps on the page.

*.img GIF files GIF-format renditions of all images in the generation, used to speed display of the generation at 100% zoom in Web View. This file exists only if there was a PARSE_RENDER_IMAGE=1 statement in server.cfg when the generation was captured.

*.Z Compressed report file If the generation has been compressed, the directory contains only a *.Z file. This file contains all information from all of the other file types.



Note For a rendition of the full generation, all html or pdf files are found in the generation directory or its frame subdirectory; for renditions based on a page security, they are found in the subdirectory for the page security or its frame subdirectory. For page securities which “Grant access to all pages,” renditions are placed in the generation directory, not in a separate subdirectory.

Directory or File Description Notes

frame Files for frame-based HTML rendition

Contains HTML files for table of contents and for each page of a frame-based HTML rendition. May also contain gif files for graphic objects in the rendition.

id Files for rendition based on page security

id is the page security ID. This subdirectory may contain files for any rendition type—single-file HTML, frame-based HTML, text, or PDF. If there is a frame-based rendition for the page security, there is an id/frame directory to contain the files.

*.pdf PDF file A single PDF file containing all the pages of the generation or page security; the only file for a PDF rendition.

*.txt Text file A single text file containing all the pages of the generation or page security; the only file for a text rendition.

*.html HTML file A single HTML file containing all the pages of the generation or the page security; the only file for a single-file HTML rendition.

*_o###.gif GIF file A GIF image file; ### is the object id of the display item as stored in the object file (*.o) for the generation; HTML rendition files may refer to gif files for images.

*Page###.html HTML page file An HTML file containing a single page of the report generation; ### is the page number. A separate file is created for each page in a frame-based rendition.

*toc.html Table of contents file A table of contents for a frame-based HTML rendition.

*Imagepage-xxx.gif Image file Image file for HTML rendition, referenced by appropriate html file; page is the page number the image is found on; xxx is the sequential number of the image on the page.

The server.debug.log Message File


The server.debug.log Message FileThe server.debug.log file contains error and other messages generated by the Vista Plus software. If a problem occurs, the contents of server.debug.log can help you and Vista Plus customer support diagnose and correct it. server.debug.log is a plain text file and is created in the Vista Plus server installation directory.

Some messages are more critical than others; Vista Plus assigns each message one of six severity levels: Debug, Info, Log, Warning, Error, or Fatal. You can select which messages are recorded in server.debug.log by using the LOGGING statement in the server.cfg file. There are three possible settings for LOGGING:

• OFF — Logs messages of level Warning, Error, and Fatal. server.debug.log always records these messages.

• ON — Logs messages of any level except Debug.

• VERBOSE — Logs messages of any severity level.

During normal operation, most installations leave LOGGING set to OFF. You would want to set it to VERBOSE in these situations:

• Before upgrading your version of Vista Plus. It can be very useful to have detailed logging information in case of any problems during the upgrade.

• If you are having any problems with Vista Plus, especially intermittent problems whose cause is not apparent.

• Any time you are asked to do so by Vista Plus customer support.

Circular LoggingThe server.debug.log file can grow quickly—very quickly if LOGGING is set to VERBOSE—and, if you do not regularly clear it, can eventually consume a great deal of disk space. To avoid this, you can use the server.cfg statement CIRCULAR_DEBUG=size. Size is the maximum size, in bytes, of server.debug.log. When server.debug.log reaches this size, it will begin overwriting the oldest entries.

Once server.debug.log has filled up and is overwriting old entries with new ones, there is a “wrap point” somewhere in the middle of the file: the place immediately following the newest message where the next message will be written. Since messages are not all the same length, this point may be in the middle of a line, with the newest message followed by the end of the oldest message. The wrap point is marked by a | character.

Here are some other characteristics of circular logging:

• The size in the CIRCULAR_DEBUG statement is in bytes. Messages in server.debug.log can be up to 100 characters or longer; keep this in mind when deciding on a maximum size.

• The CIRCULAR_DEBUG size limit is not exact. When full-grown, the file will exceed this limit by one line (probably around 70-100 bytes).

The server.debug.log Message File


• Setting CIRCULAR_DEBUG to 0 or leaving it out of server.cfg turns off circular logging; server.debug.log will grow until you clear it manually.

• You can change the CIRCULAR_DEBUG setting at any time; to make the change take effect you must stop and restart the Vista Plus server, as described in the Vista Plus Server Administration Guide.

• When CIRCULAR_DEBUG is on, the first line of server.debug.log contains <circular>. If circular logging is off, the first line contains <.linear.> (there is no special first line if you have never used circular logging).

External Password Authorization


External Password AuthorizationYou can instruct the Vista Plus server software to use an external authentication routine to check the user name and password when a user attempts to log in; you set this option individually for each user, as described in the Vista Plus Server Administration Guide. To make this option work, you must write your own external routine which accepts the user name and password from Vista Plus and returns an authentication code indicating whether the entries are acceptable.

Note While some users may use external authentication while other users use a different type; all users who use external authentication must use the same authentication routine. You cannot set different external routines for different users.

On both UNIX and Windows servers, this routine should be a dll file. The file must:

• Accept the user name and password passed to it by the Vista Plus server software.

• Validate the user name and password, then return to the Vista Plus server a Boolean value of True if the user should be allowed to log in, or False if the login request should be refused. It can validate the name and password entries itself, or pass them on to a third-party security application for the actual authentication.

On request, Vista Plus technical support can send you a sample authorization module. You can then use this sample as a basis for your own routine. We recommend you use these steps when creating an authorization module:

1. Unpack and build the sample authorization module.

2. Install the sample authorization module on a development machine, where a security breach would cause no harm.

3. Test the sample authorization module and familiarize yourself with the code.

4. Modify the sample, developing your own module.

Warning Do not install the sample authorization module on a live Vista Plus server! Install your own authorization module on a server with live data only after it is completely developed, debugged, and tested. If the authorization module is faulty, it could allow anyone to log into Vista Plus.

When you choose to use an external authorization module, you are taking responsibility for the security of your Vista Plus database, and you do so at your own risk. Any breach of security caused or aided by the use of any external security software is not the responsibility of Open Text Corporation.

Treat this document and the sample authorization module as you do other sensitive or company-confidential material. Misuse of the information in this document or the sample authorization module could jeopardize the security of your Vista Plus data.



Tip The sample module is written in C. You must use this code, modified as described below, as the interface between Vista Plus and your authentication routine. If you pass the user information on to another routine, it can be written in any language.

About the Sample Authorization ModuleThe sample module consists of a kit that can be used to create a simple authorization plug-in. The plug-in it creates accepts any user who enters a password of SAMPLE and denies access if any other password is entered.

The compressed sample file contains these files and directories:

Sample code authorization.c

AuthSample.cpp

Actual sample code

Stub for sample code, required only for Windows compiles

Required include files DLL.h

UserAuth.h

Include files

Make files AuthSample.dsp

Makefile.SunOS

Makefile.AIX

Makefile.HP-UX

Project file when compiling for Windows

Makefile for Sun Solaris compiles

Makefile for AIX compiles

Makefile for HP-UX compiles



Building the Sample CodeAfter downloading the sample file to your disk and expanding it, follow one of the procedures below to build the sample authorization module.

To build under UNIXUse cd to change to the sample directory, then type the command to build the module for your operating system. These are the three possible commands:

make -f Makefile.HP-UX make -f Makefile.SunOS make -f Makefile.AIX

This creates the intermediate file authorization.o and the final file AuthSample.dll in the sample directory.

The makefiles support three different targets:

• clean — Deletes any existing AuthSample.dll and authorization.o files; does not create new ones.

• new — Deletes any existing AuthSample.dll and authorization.o files; compiles source files to create new AuthSample.dll and authorization.o.

• all — If the source files have changed, deletes any existing AuthSample.dll and authorization.o files; compiles source files to create new AuthSample.dll and authorization.o. This is the default.

To use one of these targets, add it at the end of the make command. For example:

make -f Makefile.HP-UX new

To build under Windows1. Launch AuthSample.dsp under Microsoft® Visual Studio.

2. Set the Active Configuration to either Release or Debug.

3. Select Rebuild All.

When finished, the result will be one of the following, depending on your choice in step 2:

• .\Release\AuthSample.dll if you chose Release

• .\Debug\AuthSample.dll if you chose Debug



Installing a Plug-in Security ModuleInstalling the sample authorization module, or your own after you have created it, is easy. Follow these steps:

1. Place the dll file—AuthSample.dll for the sample module—in the desired directory. This can be any directory; the bin subdirectory of the Vista Plus server installation directory is often a good choice. For example, on a Windows server using the default installation directory, the file would go in c:\Program Files\Open Text\Vista Plus\Vista Plus Server\bin.

2. Add a line to server.cfg that points to the new module. In the above example, it would be:

AUTHENTICATOR=C:\Program Files\Open Text\Vista Plus\Vista Plus Server\bin\AuthSample.dll

3. If your server runs Windows, stop and restart the Vista Plus Server service. On a UNIX server, stop and restart the Vista Plus server process.

Vista Plus will now use the authentication module for any user with User Supplied Authorization Method selected in the user record.

Warning To protect Vista Plus security when using a custom module, make sure only authorized users can change or install the authorization module. Specifically, make sure:• The server.cfg file is protected against modification, to prevent anyone from

changing the AUTHENTICATOR statement to point to a different module.

• Your authorization module is protected against modification and replacement.

Using the Sample ModuleThe sample authorization module is deliberately simple. It serves only as an example of how an authorization module plugs into the Vista Plus Server. If the user provides a password of SAMPLE (it must be all upper case), access is allowed. If the password is anything else, access is denied.

Before writing your own module, we suggest you test the sample module. Create Vista Plus users with differing authorization methods and see what happens when you log in using different types of passwords.

Making a Custom Authorization ModuleOnce you are familiar with when and how passwords are validated by a plug-in authorization module, you can design your own. Among other benefits, using your own module could provide one or more of the following:

• Stronger security



• Simpler administration by using an existing security subsystem to validate users

• A log of user log-in attempts, both successful and unsuccessful

The heart of the plug-in authorization module is the routine IsNamePassValid() in authorization.c. It takes a string for the name and a string for the password, and returns a Boolean value indicating whether the combination of name and password is valid. TRUE indicates that the user is valid; FALSE denies access. The routine in the sample module, shown below, merely checks for a password of SAMPLE.

/* * IsNamePassValid() * * This routine is the heart of the plug-in authorization system. Given a * user name and offered password, the routine validates the combination and * returns a Boolean:.* TRUE = the combination is correct * FALSE = the combination is incorrect, invalid, outdated, etc. */BOOL USER_AUTH_API IsNamePassValid(

const char*UserName,const char*Password)

{BOOL bUserIsValid;

/* * Put your code here! * This sample accepts any logon that uses a password of

"SAMPLE". */bUserIsValid = (0 == strcmp(Password, "SAMPLE"));

return bUserIsValid;} /* IsNamePassValid() */

To create your own authorization module, insert your validation code where indicated in the sample. This code may validate the entries itself, or it may pass the name and password on to a separate security program. If it passes the information on to another program, it must also accept a returned Boolean value and pass that value back to Vista Plus.

After compiling, be sure to test your module thoroughly; any mistakes or “holes” in your code could give unauthorized users access to any or all Vista Plus features and information. How to install your module so Vista Plus uses it is described in “Installing a Plug-in Security Module” on page 22.

The Migration Process


The Migration ProcessThe Vista Plus migration processes (VMIdentify and VMTransport) move report generations from online disk volumes to offline disks or tapes. This section describes how this is done, the file naming conventions used when storing generation files on an offline volume, and temporary files and directories VMTransport creates and uses.

This section does not describe how to use VMIdentify or VMTransport. For instructions on using them and information on migration rules, please see the Vista Plus Server Administration Guide.

The Migration ProcessMigrating report generations from online to offline is a two-step process. First you run VMIdentify to identify the report generations to be archived—or compressed or deleted—then you run VMTransport, which actually removes the file(s) for the generation from the online volume and writes the generation on the offline volume or device. On a UNIX host, this can be any device; on a Windows host, it can be any mapped drive letter.

VMIdentify identifies the generations which are due for a migration action based on migration rules assigned to the various Vista Plus reports. For example, a migration rule may say to migrate any generations of a report which have not been viewed by any user for at least 30 days. The rule also includes what offline disk volume or other device the generations should be moved to.

For each type of migration action (migrate, compress, and so on), VMIdentify creates a lis file in the vmfiles subdirectory of the Vista Plus server installation directory. Each lis file contains a header line followed by the generation ID of each generation for which that action should be performed, one ID per line. There is no other information in the file. The files which may be created are:

File Name Contains

CompressGens.lis Generations to compress

migrate_devID.lis Generations to migrate to this device; ID is the device ID assigned by Vista Plus when the device was created

migrate_volID.lis Generations to migrate to this volume; ID is the volume ID assigned by Vista Plus when the volume was created

DeleteOnline.lis Online generations to delete; information about the generation is also removed from Vista Plus database

DeleteRestored.lis Previously archived and restored generations to delete from online storage; offline copy is not deleted

DeleteOffline.lis Generations to delete from offline storage; information about generation is also removed from Vista Plus database



VMIdentify creates a particular lis file only if it finds generations to perform that action for.

VMTransport reads each lis file to see what generations each action should be performed for, and performs the actions: compressing generations, moving the generations marked for migration to the specified disk or tape, and so on. During this process, it creates temporary files as needed; see “VMTransport’s Temporary Files” on page 26. It also updates the Vista Plus database, noting the new location of each offline generation and removing information about deleted generations. It processes the lis files in the order given above; if there are multiple files of a type—for example, migrate_vol50 and migrate_vol51—it processes them in order by device or volume number.

Note While the lis files are generally created by VMIdentify, they do not have to be. VMTransport will read any text file with the correct name and containing a list of generations. This means you can, if there is a need to, create a list of files for VMTransport “by hand.”

Note When restoring a generation, the generation file must be on the same offline volume it was migrated to. If the file has been moved to another volume—for example, by an archiving program outside Vista Plus—VMTransport will not be able to find it and restore the generation.

Offline Volume StructureWhen VMTransport moves report generations to an offline volume, it compresses each generation into a single file and writes the file to the directory defined as the root path for the volume. The file name format is:

• On a UNIX host: Genid.tar.z

• On a Windows host: Genid.zip

id is the ID of the generation being archived. For example, when you migrate generation 4242 on a UNIX host, it creates Gen4242.tar.z in the root path of the offline volume.

Offline volumes do not contain a directory structure similar to that used in online volumes of the report warehouse; all files are created in the root path. See page 14 for a description of the directory structure of online volumes.

Note When moving a generation from one online volume to another, VMTransport simply moves the generation directory into the directory structure on the destination volume. It does not compress the generation. The structure of online directories is described on page 14.

DatabaseDelete.lis Offline generations to remove from Vista Plus database; generation file remains in offline storage, but cannot be accessed from Vista Plus; normally used for generations archived to tape

restore.lis Offline generations to restore to online volumes; restoration is requested from Vista Plus viewer clients; VMIdentify cannot create a restore.lis file

File Name Contains



VMTransport’s Temporary FilesWhen migrating generations to an offline volume, VMTransport first compresses them. This compression is a two-step process. The first step occurs in the generation directory; the second is done in the temporary directory defined by the TMP or TMPDIR statement in the server.cfg file (see page 12):

Note In this section, GID always indicates the generation ID and RID always indicates the report ID.

1. First, VMTransport compresses all of the files in the generation directory into a single file, also in the generation directory, called GID.zip.

2. It then creates two subdirectories of the temporary directory defined by TMP or TMPDIR. These directories are called vmt_n\GenGID and vmt_n\RptRID, where n is a number assigned by the operating system.

VMTransport creates an additional compressed file in each of these directories:• In vmt_n\GenGID, it creates GenGIDUnload, containing the information from

the generation record in the Vista Plus database.• In vmt_n\RptRID, it creates RptRIDUnload, containing information from the

report record in the Vista Plus database.

3. Next, it copies the GID.zip file from step 1 into the vmt_n\GenGID directory.

4. Finally, in the temporary directory, it creates a single compressed file, called GenGID.zip, containing the vmt_n\GenGID and vmt_n\RptRID directories. This is the file that is actually moved to the offline volume.

5. If the generation is being migrated to on offline tape volume, the GenGID.zip file is copied to another subdirectory of the temporary directory, called vmt_n\VolID, where VolID is the volume ID of the destination volume. All of the generations being migrated to the volume are then copied to it from vmt_n\VolID, in one operation.

For example, for generation 789, belonging to report 321, VMTransport would create these files:

• In the generation directory, 789.zip

• In the temporary directory, the subdirectory vmt_n\Gen789, containing the file Gen789Unload. It would also copy 789.zip to the vmt_n\Gen789 directory.

• In the temporary directory, the subdirectory vmt_n\Rpt321, containing the file Rpt321Unload.

• In the temporary directory, Gen789.zip, containing the vmt_n\Gen789 and vmt_n\Rpt321 directories.

It then copies Gen789.zip is then copied directly to an offline disk volume, or to the vmt_n\VolID temporary directory for an offline tape volume.



Note For the compress migration action, VMTransport only performs step 1 of this process. It compresses all of the files in the generation directory into a single file, but it does not take any information from the database.

During a restore, the process is reversed; the generation’s compressed archive file is copied from the offline volume to the temporary subdirectory, where it is uncompressed and the uncompressed files are copied back to the report warehouse.

All temporary directories and files created by VMTransport are removed when the current processing option is complete for all generations.

Note VMTransport does not create any temporary files when moving a generation from one online volume to another online volume. It merely moves the generation files directly to the destination volume.

Other Migration Temporary FilesVMIdentify and VMTransport create two other temporary files, one in the Vista Plus server home directory and one in the server’s vmfiles subdirectory:

• Both VMIdentify and VMTransport create the file VMITsync.lck in the server home directory when they begin running, and delete it when they finish. While this file exists, you cannot start either VMIdentify or VMTransport—this keeps you from running both programs at the same time, or running two instances of either one. If a problem occurs and VMIdentify or VMTransport is interrupted, you may need to delete this file by hand.

• When it finishes, VMTransport creates the file VMTrsync.lck in the vmfiles subdirectory. While this file exists, you cannot start VMTransport without specifying a specific list file for it to use (see the Vista Plus Server Administration Guide). When VMIdentify runs, it deletes this file.

Both VMITsync.lck and VMTrsync.lck are 0-byte files with no content.

Finally, VMTransport also uses the generation-specific act files in the Vista Plus server’s .locks subdirectory. As it processes each generation it increments and decrements the count, or creates and removes the act file, as necessary. See page 13 for more information.

Font Support in Web View When Using Xvfb


Font Support in Web View When Using XvfbAs described in the Vista Plus Server Installation Guide, Web View must be started in a graphics environment. If it is not, it will not be able to display graphics, and will not be able to display any report in layout mode.

As one possible solution to this, if the Web server host does not have a graphics environment available, the Installation Guide suggests using a virtual X server such as Xvfb (X server Virtual Frame Buffer) to provide a graphics environment for Web View. Xvfb supports only a limited set of fonts. When the Web server uses Xvfb to provide a graphics environment, Web View can display only fonts which are supported by Xvfb. If a report contains fonts which Xvfb does not support, a user’s browser may “hang” when they attempt to view the report in Layout mode, or the report may not display correctly.

There is nothing Web View can do about this situation: if a font is not supported by the graphics environment it is running in, Web View cannot display it. To avoid this problem, run Web View on a Windows host or, on a UNIX host, start it using OpenWin or CDE to give it a graphics context.

Using Vista Plus Utilities


Using Vista Plus UtilitiesVista Plus includes two utilities to help you find and fix problems that could occur if the Vista Plus database and the generation files in the report warehouse get “out of synch.” These are check_gens—the check generations utility, and del_gens—the delete generations utility.

The check_gens Utilitycheck_gens compares the contents of the Vista Plus report warehouse and the Vista Plus database, looking for inconsistencies. Specifically, it will find and correct these conditions:

• Empty generation directories in the report warehouse.

• Generation files and directories in the report warehouse that don’t have corresponding records in the database.

• Mismatches between the compression flag in the database and the generation files in the report warehouse.

• Generation files which are incorrectly in the top-level directory of the report warehouse.

• Generations with an original file size of zero in the database.

• Index files in the report warehouse without corresponding entries in the database.

You can run check_gens just to report on these conditions, without making any changes, or add options to correct some or all of them, as described below.

Note In general, you should use check_gens only when recommended by, and under the supervision of, Vista Plus Technical Support.

check_gens Command Formatcheck_gens [-c] [-e] [-g] [-i] [-p] [-s] [-nsize] [-Ppath] [-D] [-v] [-h]

Option Meaning

-c Fix compression flags so they accurately reflect the state of the generation files in the report warehouse.

-D Verbose mode; includes more detail in the log files.

-e Remove empty directories from the report warehouse.

-g Remove report warehouse files for generations without records in the database.

-h Display the list of options.

-i Remove index files without records in the database.

-n Retrieve size generation records from the database at one time. If you do not include this parameter, the number of records retrieved at once is set by the VMIDENTIFY_CHUNKSIZE statement in server.cfg.



To run check_gensWarning Before starting this process, make sure VMTransport is not running. If it is,

wait for it to finish before stopping the Vista Plus server.

1. If the Vista Plus server is running on UNIX, make sure you are logged in as the user who owns Vista Plus. On a Windows host, open a DOS command window.

2. On UNIX, change to the Vista Plus home directory.

On Windows, change to the Vista Plus bin subdirectory.

3. Stop the Vista Plus server. This disconnects any clients and stops any capture or migration processes that are running:

• On UNIX:./vista_service –t

• On Windows:vista_service –t

4. Open the server.cfg file and change the LOGGING setting to VERBOSE. Save the change and close the file.

5. Run check_gens in scan mode; this looks for possible problems in the generation table:

• On UNIX:./check_gens

• On Windows:check_gens

You can include the -n and/or -D options if desired.

6. When check_gens finishes, review its log files:

more check_gens.debug.logmore check_gens.log

-P Check only generations in the directory specified by path. Path must be the complete path to a directory in the report warehouse. This lets you run check_gens for a single generation or only certain generations. If you have multiple online volumes, be sure to enter the path for the correct one. The directory structure of the report warehouse is described on page 14.

-p If any generation files are stored in the top-level directory of the report warehouse, move them to the correct directory.

-s If the original file size for a generation is zero, make it non-zero.

-v Display the check_gens version.

Option Meaning



7. If there are errors in the log files, you can run check_gens again, using one or more of the repair options:

Note Important! If you have any questions or concerns about the contents of the check_gens log files¸ e-mail a copy to technical support for review before using any of the repair options.

8. After finishing with check_gens, be sure to change LOGGING back to its usual setting in server.cfg, then restart the Vista Plus server:

• On UNIX:./vista_service –s

• On Windows:vista_service –s

The del_gens UtilityLike check_gens, del_gens compares the contents of the Vista Plus database and the report warehouse. Specifically, del_gens looks for cases where there is an entry for a generation in the database but there are no files for that generation in the report warehouse. It can also look for “orphaned” generation entries: generations listed for a report that does not exist in the database. When it finds one of these inconsistencies, it can remove the entry for the generation from the database.

del_gens checks only that the directory for the generation exists and has at least one file in it. It does not verify that all the files that should exist for a generation are present. Except when looking for generations which have no report, it does not check offline generations.

Note In general, you should use del_gens only when recommended by, and under the supervision of, Vista Plus Technical Support.

Error Type Option to Use

Generation XXX not found in Database check_gens –g

Fix generations paths check_gens –p

Improper compression flag for Generation XXX check_gens –c

Remove obsolete indexes check_gens –i

Located empty directory check_gens –e



del_gens Command Formatdel_gens [-gstart-id] [-zend-ID] [-o] [-d] [-nsize] [-D] [-v] [-h]

To run del_gensWarning Before starting this process, make sure VMTransport is not running. If it is,

wait for it to finish before stopping the Vista Plus server.




3. Stop the Vista Plus server; this disconnects any clients and stops any capture or migration processes that are running:

• On UNIX:./vista_service –t

• On Windows:vista_service –t

4. Open the server.cfg file and change the LOGGING setting to VERBOSE. Save the change and close the file.

5. First, run del_gens in display-only mode to see what generations may need to be deleted:

del_gens -d [-o]

Include -o if you want to look for orphaned generations (ones whose report does not exist in the database) rather than for generations that don’t have files in the report warehouse.

Option Meaning

-d Display qualifying generations without removing

-D Verbose output—log file includes each directory that was examined

-g Remove obsolete generations with IDs above the start-ID number

-h Display the list of options

-n Report list chunk size

-o Remove generations that have no report; this option checks both online and offline generations

-v Display version information

-z Remove obsolete generations with IDs below the end-ID number



6. After reviewing the results of del_gens -d, you can run one of these commands to delete unneeded generation records:

• To remove the database records for generations that do not have files in the report warehouse:

del_gens [-g startID] [-z endID]

Include -g startID to remove records only for generations with an ID above startID. Include -z EndID to remove records only for generation with an ID below EndID. You can include both to remove generations only within a specific range of IDs.

• To remove the database records for orphaned generations:del_gens -o

7. After finishing with del_gens, be sure to change LOGGING back to its usual setting in server.cfg, then restart the Vista Plus server:

• On UNIX:./vista_service –s

• On Windows:vista_service –s

Tip del_gens -o deletes only the database record for orphaned generations. There could be files for those generations left in the report warehouse. We recommend you run check_gens to look for this and delete any unneeded files.

Adding Index Values to the Database


Adding Index Values to the DatabaseStarting with version 5.5, Vista Plus can store index values for captured generations in the database as well as in the generation directory in the report warehouse. Having the index values in the database can greatly speed up global index searches and searches across multiple report generations, and also allows you to include offline generations in these types of searches.

Unless the server.cfg file includes the statement StoreIndexValuesInDB=0 to turn the feature off, Vista Plus 5.5 and later add index values to the database when a generation is indexed during or after capture. However, generations captured before this feature was available have index values stored only in the report warehouse. You can add these existing index values to the report warehouse using the convert_gens utility.

convert_gens reads index values from the Vista Plus report warehouse and, if they are not already there, inserts them into the Vista Plus database. It can do this for all generations, only those captured after a certain date, or only a certain number of generations (for example, the 50 most recent) for each report.

convert_gens only copies existing index values from the report warehouse to the database. It does not perform a re-index based on current index definitions. Also, since it uses values from the report warehouse, it works only on online generations. To add index values for an offline generation to the database, you must restore the generation. After you run convert_gens, you can delete or remigrate the generation and it will remain searchable.

Depending on the number of generations and index values involved, convert_gens can take a considerable time (up to hours or even days, depending on the number of index values) to add the index values to the database.

Note In most cases, if you are going to add index values to the database, StoreIndexValuesInDB should either not be present in server.cfg, or should be set to 1. If it is present and set to 0, index entries for newly captured generations are not written to the database. The database would contain entries only for the generations you converted with convert_gens and not for others, which could mean that global index or multi-generation searches would miss some matches.



Space ConsiderationsAdding index values to the Vista Plus database increases the size of the database. However, in the long term, it may either increase or decrease the total space used by Vista Plus.

The amount of space needed in the database depends on the number, type, and size of the defined indexes and the number of generations you are converting. You can estimate the exact amount of extra space running convert_gens will use by calculating the total size of all the index files for the generations you’re converting. These files are in the Vista Plus report warehouse and have a .i extension. In most cases, the amount of data added to the database is approximately twice the total size of all the .i files. (In addition to the actual index values, convert_gens creates a key structure to speed searching, thus requiring the extra space.) For text (as opposed to numeric) indexes, the requirements could be even higher, up to three times the .i file size; short text indexes use relatively more space than longer ones.

If you want to see how much space the .i files use so you can calculate how much space convert_gens will use, you can add the size of the .i files for all of the generations you will be converting. However, if any of those generations are compressed, the .i files are contained in the compressed generation file. In that case, to estimate the total size of the .i files, you could open a typical generation (which uncompresses its files), see the size of its .i files, then multiply that by the number of generations.

Despite this size increase in the database, storing index values in the database could decrease total Vista Plus disk space usage by decreasing the space needed for the report warehouse. It does this in two ways:

• With index values in the database, Vista Plus does not have to uncompress each generation when you do a generation search or a global index search. A generation is only uncompressed if you select and open it from the list of search results.

• With index values in the database, you can search offline generations. This may let you move generations offline sooner, freeing space in the online volumes.

Whether the space saved in the report warehouse offsets the additional space required in the database depends on your specific circumstances.



convert_gens Command Formatconvert_gens [-c] [-d zzz] [-l] [-n xxx]

Note If you want to stop convert_gens while it is running, press one of these keys:• p to pause the process but not exit the program.• r to resume after pausing.• x to cancel. Any values already added to the database remain there. A

message tells you how many generations have been converted.

These options are available only while index values are being written to the database, not while convert_gens is compiling the list of generations to convert. This means they have no effect if you included the -l option.

Running convert_gensThere are two ways to use convert_gens: you can use one command to convert all existing online generations, or you can first create a list of the generations to convert, optionally limited by date or number of narrations per report, then convert the listed generations using a second command.

Tip You may want to run del_gens before convert_gens to check for missing generation files. convert_gens cannot convert a generation which is in the Vista Plus database but does not have files in the report warehouse. If it finds one, it writes an error to server.debug.log.

To convert all generations1. If the Vista Plus server is running on UNIX, make sure you are logged in as the user

who owns Vista Plus. On a Windows host, open a DOS command window.



3. Type this command:

convert_gens

Tip On UNIX, include ./ before the command.

Option Meaning

-c Convert the generations listed in the GensToConvert.txt file. Use after -l.

-d List only generations captured in the last zzz days. Do not include older generations. Works only with -l.


-l List generations selected for conversion in the file GensToConvert.txt. Do not write index values to the database.

-n List only the most recent xxx generations for each report. Do not include older generations. Works only with -l.



4. convert_gens displays a message warning that it’s about to make database changes and asks if you want to continue. Type y and press Enter to continue or just press Enter to exit the program without converting any generations.

The conversion may take a considerable time, depending on the number of generations. You can use the p, r, and x options while it is running, as described above. When convert_gens finishes, it tells you how many generations were converted.

Note Important! Do not run two instances of convert_gens at the same time. This is not supported.

To list generations, then convert themTip On UNIX, include ./ before all commands.




3. Type this command:

convert_gens -l [-n xxx] [-d zzz]

This lists—in the file GensToConvert.txt—the generations to be converted. The list includes the generation ID, report name, generation description, and the name of the captured file.

If included, -n xxx lists only the most recent xxx generations for each report. If a report has more than this many online generations, additional ones are not included.

If included, -d zzz lists only generations captured in the last zzz days. Older generations are not included.

4. To read the list from the file and perform the conversion, type this command:

convert_gens -c

5. convert_gens displays a message warning that it’s about to make database changes and asks if you want to continue. Type y and press Enter to continue or just press Enter to exit the program without converting any generations.

The conversion may take a considerable time, depending on the number of generations. You can use the p, r, and x options while it is running, as described above. When convert_gens finishes, it tells you how many generations were converted.

Note Important! Do not run two instances of convert_gens at the same time. This is not supported.

Removing Index Values from the Database


Removing Index Values from the DatabaseAs discussed in “Space Considerations” on page 35, storing index values to the Vista Plus database in addition to the report warehouse can greatly increase the size need for the database. For this reason, you may change your mind and decide that the benefits of having the index values in the database are outweighed by the extra space needed. If this happens, you can set StoreIndexValuesInDB=0 in the server.cfg file so future captures and indexing operations do not create entries in the database. You can also use the revert_gens command to remove existing index entries from the database.

Warning Before changing the StoreIndexValuesInDB setting or using revert_gens, contact Vista Plus Technical Support. You should be certain of your decision before making the change. Changing the StoreIndexValuesInDB setting from 1 to 0, then back again at a later time can cause data inconsistencies that can hurt Vista Plus performance and potentially lead to other problems.

After you use revert_gens, you or your database administrator will need to perform further actions to reclaim the space that had been used by the database index entries. How to reclaim the space is described after the revert_gens command description.

revert_gens removes index entries only from the Vista Plus database. The index entries in the report warehouse are not affected. As long as the IndexSearchMethod parameter in server.cfg is set to either file or auto, all index searches will continue to work after you run revert_gens.

revert_gens Command Formatrevert_gens [-c] [-l] [-d xxx] [-n zzz] [-h]

Option Meaning

-c Revert the generations listed in the GensToRevert.txt file. Use after -l.

-d List only generations captured in the last xxx days. Do not include older generations. Works only with -l.


-l List generations selected for conversion in the file GensToRevert.txt. Do not write index values to the database.

-n List only the most recent zzz generations for each report. Do not include older generations. Works only with -l.



Using revert_gensTip In almost all cases, before you use revert_gens, you should make sure the

server.cfg property StoreIndexValuesInDB is set to 0. If it isn’t, index values will continue being added to the database for new generations and generations that are re-indexed.

There are two basic ways to use revert_gens:

• Run it without the -l option to find and immediately revert all generations that have index entries in the database.

• Run it first with the -l option to create a list of generations that need to be reverted, then run it with the -c option to remove the index entries from the database. With this method, you can use the -d or -n option to limit the generations affected.

Here are some example commands:

• revert_gens

Removes all index entries for all generations from the database.

• revert_gens -l

Lists all generations which have index entries in the database. The list is in the file GenstoRevert.txt.

• revert_gens -l -n10

Creates a list in GensToRevert.txt of generations with index entries in the database. The list include only the most recent ten generations for any report. If a report has more than ten online generations, older generations are not listed.

• revert_gens -c

Removes from the database the index entries for the generations listed in GensToRevert.txt.

Note If you want to stop revert_gens while it is running, press one of these keys:• p to pause the process but not exit the program.• r to resume after pausing.• x to cancel. Any values already removed from the database remain

deleted. A message tells you how many generations have been reverted.

These options are available only while index values are being deleted from the database, not while revert_gens is compiling the list of generations to revert. This means they have no effect if you include the -l option.



Reclaiming the Database Spacerevert_gens removes the index entries for the affected generations from the Vista Plus database tables. However, to reclaim that space so it can be used for other purposes requires additional actions at the database level. The procedure to do this differs depending on whether you use Oracle or MySQL for the Vista plus database.

Reclaiming the Database Space in OracleThe Oracle database software includes several different ways to reclaim unused space in data files, such as export/import, manual tablespace reorganization, and Enterprise Manager tablespace reorganization. Your Oracle DBA can decide which method to use and perform the appropriate procedure to reclaim the space.

Reclaiming the Database Space in MySQLHow to free the database space previously used by the index entries depends on whether the MySQL my.ini file sets innodb_file_per_table to 1 or 0.

Note Either of these procedures should be performed by the MySQL database administrator.

If innodb_file_per_table is set to 1, follow this procedure:

1. Use the mysql command to start MySQL:

mysql -u name -p password

Name is the name of any MySQL user with permission to read and write the Vista Plus database. Password is the password for that user.

2. At the MySQL prompt, enter:

OPTIMIZE TABLE vp_index*

This optimizes all Vista Plus index tables. This could include some tables which were not affected by the revert_gens command, and therefore have no space to be reclaimed. Optimizing these tables does not do any harm. However, if you want to optimize only the tables affected by revert_gens, you would need to look at the list of affected generations, determine which reports those generations belong to, and see what indexes are defined in those reports. You could then look in the index definition table to see the table names for each index and optimize only those tables. It is generally easier to optimize all the index tables.



If innodb_file_per_table is not set to 1, the only way to make the space available is to dump all the data from the ibdata file, delete the file, then create a new file and restore the data to it. Follow this procedure:

1. Stop the Vista Plus server

2. Use mysqldump to dump all your InnoDB tables:

mysqldump -u root -p password -A > dump.sql

Password is the password for the MySQL root user.

3. Stop MySQL server.

4. Remove all the existing tablespace files, including the ibdata and ib_log files. If you want to keep a backup copy of the information, copy all the ib* files to another location before the removing the files in your MySQL installation.

5. Remove any .frm files for InnoDB tables.

6. Edit the MySQL my.ini file and set innodb_file_per_table to 1.

7. Configure a new tablespace.

8. Restart MySQL server.

9. Import the dump files.

mysql -u root -p password < dump.sql

Password is the password for the MySQL root user.

10. Restart the Vista Plus server.

Tuning Vista Plus Server Configuration


Tuning Vista Plus Server ConfigurationMany configuration settings, both within Vista Plus and at the operating system and database levels, interact to affect many aspects of Vista Plus performance: memory and other system resource use, general response time,. search performance, and more. Adjusting some of these settings can also help avoid certain types of errors caused by exceeding set limits or running out of allocated resources. The following sections describe several areas where you may be able to change configuration settings or resource allocation to improve Vista Plus behavior and performance. They include:

• Scalability configuration options

• Kernel parameter settings on Sun® Solaris® and HP-UX

• Adjusting MySQL memory limits

• Preventing lost connections to an Oracle database

• Improving search performance

Scalability Configuration OptionsScalability refers to Vista Plus’s ability to maintain good performance and reliability as the number of connected users and captured reports increases, without adversely affecting other applications. Correctly setting certain configuration options—during installation, in the server.cfg file, or in the warehouse configuration—can help ensure the best performance in your installation, and can minimize the network and server resources Vista Plus uses.

There are two areas you can configure which affect scalability: the way in which connections between clients and the Vista Plus server are handled and how many server processes may run at one time. Both of these areas are discussed below.

Client – Server ConnectionsTwo types of connections are possible between the Vista Plus server and any of its client programs: persistent and short-lived. A persistent connection exists for as long as the client program is connected to the server, whether or not the client and server are exchanging information. A short-lived connection is opened whenever the client requests information from the server and closed after the server sends the data back to the client. A persistent connection can give slightly better performance to the client using it, since it does not have the overhead of re-opening the connection for each information request. However, since persistent connections remain open longer, they use more resources on the Vista Plus server.



The type of connection used is set for each server by the vadmin ModifyWarehouseConfig command’s ctype parameter. You can set ctype to one of three values:

• p for persistent connections

• s for short-lived connections

• u for user-selectable

The default is for persistent connections for compatibility with previous Vista Plus releases. However, short-lived connections are generally more efficient and result in the best overall client-server performance. If conserving server resources is not an issue, you can use persistent connections, which may reduce response time for individual clients. If ctype is set to user-selectable, individual users can select short-lived or persistent connections when they log on using the Windows Client or Web View. Other clients—vadmin, Server Admin, and capture clients—use persistent connections.

When using short-lived connections you can set the amount of time before a connection is shut down using the ModifyWarehouseConfig ctimeout option. ctimeout specifies a number of seconds from 1-30; if a short-lived connection has no activity for this long, it is shut down and has to be re-opened the next time the client communicates with the server. This means that connections that are in constant communication with the server—for example, when a file is being sent to the server to be captured—are not shut down and re-opened between every packet of data. Instead, they stay open until there is a period of time with no data being sent. For best performance, we recommend setting ctimeout to one or two seconds; client performance can degrade if it is much higher than that, especially at large installations. Setting a longer time-out value reduces the benefits of short-lived connections, as it leaves more connections open for longer periods of time; if you have a high time-out value you may need to increase the MAX_WORKER_PROCESSES setting to allow more concurrent client connections (see the next section).

Note Important! Regardless of server host resources, most UNIX servers cannot support more than 200-300 persistent connections; Windows servers cannot support more than about 450. We highly recommend short-lived connections for installations which approach or exceed these limits.

Server ProcessesNote This is a greatly simplified description of the internal structure of the Vista

Plus server software. It is included only to give some background for the server.cfg settings discussed later in the section.

The “Vista Plus server” is made up of many different pieces of software. When the server is started, what actually starts is a process called the Server Pool Monitor. This Monitor spawns child tasks as needed to handle communication between the Vista Plus server and clients. These child tasks are called Worker Processes. The Worker Processes accept information and requests from the clients and handle all communication with the clients.



As needed, they pass on requests to Service Processes, which perform server functions such as report capture, indexing, and so on.

Each Worker Process handles communication with one client process at one time. If you are using persistent client connections, the maximum number of Worker Processes limits the number of users who can connect to the server at one time. If you use short-lived connections (see the previous section), more users can be active, since not all users will actually have a connection open with the server at any one time.

When a client requests a connection with a Service Process, the request is placed in the connection queue. If there is no Worker Process available, the request waits until one is available. If no Worker Process becomes available before a certain time has elapsed—you can set the time period—the server tells the client it cannot make the connection.

In the server.cfg file, you can set these values that affect how many connections are available and how they are handled:

• The INITIAL_WORKER_PROCESSES statement sets the number of Worker Processes that the Server Pool Monitor starts when it begins. The default is three. This is also the minimum number of Worker Processes that can be running; the server will leave this many processes active even if there are no client connections.

• The MAX_WORKER_PROCESSES statement limits the number of Worker Processes that can exist at the same time. The default is 100. As stated above, if you use persistent connections, this also sets the maximum number of users who can log on to the server at once; sites using persistent connections will need to set this value higher than similar sites using short-lived connections. The time-out period for short-lived connections also affects the number of Worker Processes needed.

Large installations will need to increase MAX_WORKER_PROCESSES. You will want to increase it if users receive “cannot communicate with server” error messages more than once in a great while. Increasing the CONNECTION_QUEUE and/or CONNECT_TIMEOUT settings (described below) can also reduce the number of unsuccessful attempts to connect to the server.

• The MAX_SERVICE_PROCESSES statement sets the maximum number of Service Processes which can exist at one time. The default is three per service type. There are three types of services: action processing, indexing, and report parsing. Again, larger installations will want to increase this value; the maximum setting is 100. Since at any given time many connected clients will not be using a Service Process, this setting does not need to be as large as MAX_WORKER_PROCESSES.

• The CONNECTION_QUEUE statement sets the maximum number of client requests that can be waiting for a Worker Process at one time. The default value is 100; if you set this to 0, there is no connection queuing and the server tells the client immediately if there is no Worker Process available when the client requests one. The maximum setting is 1000.

While the CONNECTION_QUEUE does not increase the number of people who can connect to Vista Plus, it can keep users from receiving errors by holding the connection request until a Worker Process is available. In general, if your



MAX_WORKER_PROCESSES is large enough for everyone who may want to use Vista Plus, you can have a very small CONNECTION_QUEUE setting. If your system resources dictate a small MAX_WORKER_PROCESSES, you can reduce the frequency of refused connections by increasing the CONNECTION_QUEUE setting.

• The CONNECT_TIMEOUT statement determines how long a client connection request will wait in the connection queue before the server tells the client it could not make the connection. The value is in seconds; the default is 15. If you have users receiving “cannot communicate with server” error messages, you may want to increase this number as well as MAX_WORKER_PROCESSES. However, while this can decrease the number of failed connections, it also increases the length of time users wait before they see the error message and can try again. The highest allowed setting is 600 (ten minutes).

Note Before being placed in the Vista Plus connection queue, the client request is processed by an operating system queue. This queue has a fixed size which is out of the control of Vista Plus. On the AIX operating system, this initial queue can contain only ten requests. Therefore, you may want to make the Vista Plus connection queue larger on AIX systems, to prevent the operating system queue from filling up due to a lack of space in the Vista Plus queue.

Obviously, the interactions between these settings and exactly how they affect your system performance can be quite complex and depend on many factors: how much memory and other system resources are available on the server, the number of Vista Plus users you have, how often and for how long those users use Vista Plus and exactly what they do while connected, and so on. In general, larger installations will want to allow larger numbers of worker and service processes while using short-lived connections with a low time-out setting. However, as you increase the number of simultaneous connections accessing the Vista Plus database, the limiting factor for performance can switch from how many users can connect to the server to how fast the server accesses information in the database. Allowing too many connections at once may result in occasional database error messages.

For help in adjusting these settings to increase your Vista Plus performance, please contact customer support.



Recommended Kernel Parameters on the Server HostThe following sections discuss three types of operating system kernel parameters which can affect Vista Plus server operation: shared memory parameters, semaphore parameters, and segment size parameters. We discuss what these kernel settings do and how that can affect Vista Plus. Most of these settings exist on both Sun Solaris and HP-UX server hosts:

• All of the shared memory and semaphore settings are found on Solaris. They are in the etc/system file.

• A few of these settings do not exist on HP; they are noted in the tables (with ------- ) and descriptions. The settings that do exist are found in usr/conf/master.d/core-hpux, but are easiest to check and modify using the sam tool.

• The segment size parameters are found only on HP.

If you change any of the parameters described in the following sections, you must reboot the server for the new setting to take effect.

Warning The sections below discuss how these kernel parameters affect the operation of Vista Plus. Open Text Corporation does not claim that the recommended settings and guidelines below will work on all server hosts or in all configurations. All recommendations and information apply only to Vista Plus. Other software on the server host may have its own requirements for these settings; please see the documentation for your operating system and for any other software installed on the host for more information.

Note Vista Plus memory and semaphore use is the same on IBM AIX as on other versions of UNIX. However, AIX handles these areas differently; there is no need to modify any AIX kernel settings for Vista Plus.

Shared Memory ParametersThe amount of shared memory Vista Plus uses varies depending on the size of the user license and the number of report warehouses installed. Vista Plus requires four memory segments, of varying sizes:

Type Size in Bytes Description

Warehouse 44 + (24 * number of seats in the license)

There is one segment for each warehouse on the system. Vista Plus always requests enough memory for at least 1000 seats, even if fewer users are licensed; the maximum number of users it will allocate memory for is 10,000. Therefore, the minimum size of this segment is 24,044 bytes and the maximum is 240,044 bytes. If the license is for more than 1000 users but fewer than 10,000, you can use the formula to determine the size of the segment.



Note Vista Plus user licenses are based on the total number of users, not the number of concurrent users. We have found allocating memory for 10,000 concurrent users on a single host to be sufficient even in large installations.

This table shows the operating system shared memory settings (all values are decimal):

Note HP also has a shmem parameter. This turns shared memory on and off. It must be set to 1 to enable shared memory. Vista Plus will not work if shared memory is turned off.

Here are short descriptions of each setting and how they relate to Vista Plus:

• shmmax — Is the largest allowed size of a single memory segment. As discussed above, depending on the number of licensed users, the largest single segment Vista Plus requires is between 408,424 and 4,080,424 bytes. shmmax must be large enough to allow a memory segment this size.

• shmmin — The opposite of shmmax, this is the smallest allowed size of a shared memory segment. For example, if shmmin is ten and you request a two-byte segment, you will have problems. This is most commonly set to one; it must be one to ensure proper Vista Plus operation. This parameter does not exist on HP.

• shmseg — Is the largest number of memory segments any one process can have at once. As discussed above, each Vista Plus server needs only four memory segments.

• shmmni — Sets the total number of memory segments that can be allocated at once, for all processes. Each segment has a unique identifier; shmmni tracks the total number of identifiers. For most installations, 100 segments (this is the default for many Solaris systems) should be more than enough.

License 424 + (408 * number of seats)

Again, Vista Plus will request enough memory for at least 1000 seats but not allocate more memory than needed for 10,000, resulting in a segment of from 408,424 bytes to 4,080,424 bytes. Use the formula to find the actual segment size in your situation based on the number of users. There is only one of these segments, regardless of the number of warehouses.

Communication

4924 Used by Vista Plus to talk with its services; there is one communication segment per warehouse.

Tracing 40812 There is also one tracing segment per warehouse.

etc/system Setting Sun Maximum Value

HP Minimum Value

HP Maximum Value

set shmsys:shminfo_shmmax 4,294,967,295 2048 1,073,741,824

set shmsys:shminfo_shmmin 4,294,967,295 --------- ---------

set shmsys:shminfo_shmseg 32,767 1 shmmni

set shmsys:shminfo_shmmni 214,783,647 3 1024

Type Size in Bytes Description



Note On Solaris, the total size of all segments cannot be more then 25% of the total system memory. This means if you have 1 GB of RAM in your system and you request six segments of 50 MB each, you would get no more than five of the requested segments. This is true even if the shmmni and shmmax settings allow more segments and segments of 50MB or larger.

Semaphore ParametersSemaphores are basically used to make sure only one action is happening to a piece of data at a time. This ensures two processes do not try to update the same item at the same time. Each piece of data for a process can have a semaphore associated with it. To make the maintenance of the semaphores easier, they are often grouped in semaphore sets. This allows a process to carry out an action on the set, which applies it to each semaphore in the set, instead of carrying out the action separately on each semaphore.

Vista Plus uses eleven semaphores for each warehouse on a server. Vista Plus allocates semaphores in “sets” of one.

This table shows the operating system semaphore settings:

Here are short descriptions of each parameter and how they can affect Vista Plus operation.

• semmap — When the system is first booted, one big “chunk” of memory is used for semaphores. As processes request memory for semaphores, this chunk gets broken into many smaller pieces. semmap keeps track of the available parts of this memory. For example, we start with one big chunk:

AAAAAAAAAAAAAAAAAAAAAAAAAAA

etc/system Setting Sun Maximum Value

HP Minimum Value

HP Maximum Value

set semsys:seminfo_semmap 2147483647 4 32767 or (semmni + 2), whichever is less

set semsys:seminfo_semmni 65535 2 semmns

set semsys:seminfo_semmns 2147483647 2 32767

set semsys:seminfo_semmnu 2147483647 1 nproc – 4

set semsys:seminfo_semmsl 2147483647 --------- ---------

set semsys:seminfo_semopm 2147483647 --------- ---------

set semsys:seminfo_semume 2147483647 1 smmmns

set semsys:seminfo_semusz 8 * (semume + 2) See notes below

--------- ---------

set semsys:seminfo_semvmx 2147483647 1 65535

set semsys:seminfo_semaem 2147483647 0 32767 or semvmx, whichever is less



Two processes request semaphores. Each process keeps track of the memory area it has and semmap keeps track of what is free.

AAAAAAAAAAAAAAAAAAAAAAAAAAA| | | | semmap PID1 PID2

If PID2 releases its semaphores back to the pool it looks like this:

AAAAAAAAAAAAAAAAAAAAAAAAAAA| | | | semmap PID1 semmap

Since semmap is tracking two locations that are not contiguous (they don’t touch each other), it must maintain two entries, one for the first location and another for the second. The number specified in the semmap setting is the maximum number of separate locations semmap can keep track of. If the memory gets fragmented into more chunks than semmap can handle, it “loses track” of some chunks and they are unusable until the system is rebooted and the memory gets put back into one big chunk.

Some Solaris systems use a default of 10 for semmap. This will probably need to be increased for Vista Plus. Remember, this is the maximum number of semaphore chunks the system can keep track of: as processes request and release semaphores, the number of chunks can be higher than the total number of semaphores used. With a 4.3 or later server, especially with multiple warehouses, there can easily be more then ten chunks. (The system is pretty good about making two neighboring chunks into one big chunk, but it may not always be able to do this.) semmap will then not be able to track all the chunks and will essentially lose them until the system is rebooted. This can make Vista Plus seem to work for a while and then stop. The higher the value, the more fragmentation can be handled.

• semmni — Defines how many semaphore sets can be used on the system at one time. For example, if it is set to ten and two processes each request five semaphore sets, no other processes will be able to get a semaphore set until one of the processes is finished with its. Note that semmni limits the number of semaphore sets, not the number of semaphores. Since Vista Plus allocates its semaphores in sets of one, it needs as many semaphore sets as it does semaphores, so this setting should be the same as semmns.

• semmns — Defines the total number of semaphores on the system, which determines the amount of memory available for semaphores. Increasing this number increases the semaphores available in two ways: not only are more semaphores allowed, but because the size of the memory chunk is increased, the fragmentation of the chunk is decreased and, in general, bigger segments are available. This makes it more likely for semmap to be able to keep track of the available chunks and allocate the needed semaphore memory when it is requested. This is the setting most likely to need adjustment for Vista Plus, especially on servers with several warehouses installed. If you see server.debug.log messages saying the service_daemon could not start, it could be because semmns is too low. If you do need to raise this setting, don’t set it too high as



this increases the system overhead needed to keep track of semaphore memory chunks.

• semmnu — Defines the number of semaphore undo structures. Basically, if you are using a semaphore to change something, but want to be able to undo the change if something goes wrong, the undo structure records what you did and allows you to undo it. Vista Plus does not use undo structures, so this setting has no effect on Vista Plus.

• semmsl — Limits the number of semaphores one process can have. It should be less then semmns to keep one process from taking all the semaphores on the system. No single Vista Plus process uses more than three semaphores, so you are unlikely to need to change this setting for Vista Plus. This parameter does not exist on HP.

• semopm — When a process has a semaphore, it wants to do operations on that structure. It can do one command at a time or several at once. semopm limits the number of operations that can be done to a semaphore at once during a semop call. Vista Plus performs only one operation per semop call so this setting has no effect on Vista Plus. This parameter does not exist on HP.

• semume — Is the maximum number of records any single process can have in an undo structure. Vista Plus does not use undo structures, so this setting has no effect on Vista Plus.

• semmvx — Limits the maximum value of a semaphore. Never set this above 32767, even though it is possible to do so. Lower is usually not a problem unless it is extremely low. In most cases you don’t need to adjust this value for Vista Plus.

• semaem — Sets the maximum value of an "adjust on exit" undo element. Vista Plus does not use undo structures, so this setting has no effect on Vista Plus.

• semusz — Is the size in bytes for undo structures. This is set to 8 * (semume + 2) when the system is booted. Vista Plus does not use undo structures, so this setting has no effect on Vista Plus. This parameter does not exist on HP.

Segment Size ParametersOn HP-UX, the segment size parameters can affect performance of Vista Plus and the database software it uses. There are three parameters you may need to adjust: maxdsize (maximum data segment size), maxssize (maximum stack segment size), and maxtsize (maximum text segment size). We recommend setting all of these parameters to at least the recommended size for your operating system version.



MySQL Memory LimitsIn some cases, displaying an extremely long list of items from a UNIX server host in the Server Admin client causes a Server unknown error message. This happens when the MySQL user reaches its defined operating system memory limit while displaying the list. This symptom has been seen when listing page security definitions when there are tens of thousands of page securities defined.

When this happens, increase the allowed data size and virtual memory size use for the operating system user that MySQL runs under. Setting these limits to a larger value, or to unlimited, can resolve the problem.

The exact commands to use to check and change the data and virtual memory allocations vary depending on your operating system and shell. In most cases, you can use the ulimit command to do this. There are also ways other than ulimit to change the data and memory limits. Refer to your operating system documentation.

Preventing Lost Oracle Database ConnectionsIn rare cases, some Vista Plus customers have had problems where Vista Plus loses its connection to the Oracle database for no apparent reason. The server.debug.log file shows the error message ORA-03114: not connected to ORACLE.

The solution to this error and the dropped connection is outside Vista Plus. Implementing one or more of the recommendations in the DataDirect knowledge base article: 3032 : ORA-03114 not connected to ORACLE should correct the problem.

http://knowledgebase.datadirect.com/articles/Article/3032

http://knowledgebase.datadirect.com/articles/Article/3032




Index

Symbols.lis files in migration 24.locks directory 12, 13

Bbundles, temporary files 12

Ccapturing reports see report capturecheck_gens utility 29CIRCULAR_LOGGING 17client-requested archiving, temporary files 12connection type 42

timeout period 43CONNECTION_QUEUE 44CONNECTION_TIMEOUT 45convert_gens utility 34

Ddatabase size and index values 35del_gens utility 31dircapture temporary files 12

Eemulating printer settings using -x 8encryption 11

and Web View 11of passwords 11

epurposing files 15error logging levels 17etc/system file on Solaris 46external password authorization 19

process 19sample module 20warning 19

Ffiles in the report warehouse 15

epurposing 15folder structure for the report warehouse 14font support in Web View 28

Ggenerations

checking database for 29deleting 31epurposing files 15files for each 15

group link record for ImportFileInfo 3group record for ImportFileInfo 3groups, adding or deleting with ImportFileInfo 2gsr temporary files 13

HHP-UX

kernel parameters 46sam tool 46usr/conf/master.d/core-hpux 46

IImportFileInfo 2

data file format 2group link record 3group record 3user record 3

password parameter 4ptype parameter 4setting passwords 4

index valuesadding to database 34

space considerations 35removing from database 38

reclaiming space 40indexing, temporary files 12INITIAL_WORKER_PROCESSES 44

Index


Kkernel parameters 46

segment size 50semaphores 48shared memory 46

Llock manager temporary files 12LOGGING 17

MMAX_SERVICE_PROCESSES 44MAX_WORKER_PROCESSES 43, 44message logging levels 17migration 24

.lis files 24offline volume structure 25VMIdentify temporary files 27VMTransport

temporary files 26VMTransport temporary files 27

Ooffline volume structure 25operating system kernel parameters 46

PPARSE_RENDER_IMAGE 15passwords

encryption 11external authorization 19setting with ImportFileInfo 4

PCL codes, prepending with -x 8performance 42

connection type 42worker processes 43

persistent connections 42prepending PCL codes with -x 8printer settings, emulating with -x 8printing, temporary files 13

Rreport capture

temporary files 12-x option 8

file location 8samples 9

report warehouseepurposing files 15files 15folder structure 14

repository search, temporary files 13requirements

segment size 50semaphores 48shared memory 46

revert_gens utility 38reclaiming database space 40

Sscalability 42

connection type 42worker processes 43

searching, temporary files 12, 13segment size parameters 46, 50semaphore parameters 46, 48server pool monitor 43server.cfg file

CIRCULAR_LOGGING 17CONNECTION_QUEUE 44CONNECTION_TIMEOUT 45encryption flags 11INITIAL_WORKER_PROCESSES 44LOGGING 17MAX_SERVICE_PROCESSES 44MAX_WORKER_PROCESSES 43, 44PARSE_RENDER_IMAGE 15temporary directory setting 12temporary directory settings 12TMP 26TMPDIR 26

server.debug.log file 17circular logging 17

service processes 43, 44shared memory parameters 46short-lived connections 42


Index

Solarisetc/system file 46kernel parameters 46

Ttemporary files 12

.locks directory 12, 13TMP directory 12, 13TMPDIR directory 12, 13VMIdentify 27VMTransport 26, 27warehouse parameters setting 12

TMP directory 12, 13, 26TMPDIR directory 12, 13, 26

Uusers

adding or deleting with ImportFileInfo 2external password authorization 19

Vvadmin

gsr command temporary files 13ImportFileInfo see ImportFileInfo

ModifyWarehouseConfigctimeout parameter 43ctype parameter 43

vconvert temporary files 12virtual X server and Web View 28VMIdentify 24

.lis files 24temporary files 27

VMITsync.lck 27VMTransport 24

offline volume structure 25temporary files 13, 26, 27

VMTrsync.lck 27

Wwarehouse parameters

temporary directory 12Web View font support 28worker processes 43, 44

X-x option for report capture 8

file location 8samples 9

Xvfb and Web View 28

Vista Plus Technical Addendum

Documents

open text corporation

property of open text

importfileinfo file

dsexporter command format

product names

registered trademarks

dsexporter copies

output file format