DataStage Administrator commands to find: No Command Description Command 1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS; SELECT COUNT(*) FROM DS_JOBOBJECTS; 2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS 3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job' CLEAR.FILE RT_Logxx where xx is JOBNO. 4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME> 5 Find how many Jobs are using given Hashed file as INPUT select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOB DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECT DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group b 6 Find how many Jobs are using given Hashed file as OUTPU select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOB DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJEC DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group b 7 Find how many Jobs are using given Hashed file as INPUT and OUTPUT select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOB DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Ha 8 View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME 9List metadata of hashfile SELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV" "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", 10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VAL 11 Find which user modified the job last SELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLAS NAME';
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfile SELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM
DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the job last SELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using
Keymgmt function in Datastage UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables
used in datastage proj SELECT * FROM SDKSequences;
15List Datastage error message decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI: Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters 1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs <projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams <projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations <projectname> <jobname>
4Generate XML report of job , its stages and links @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -wait -jobstatus -param
<name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop <projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import <projectname> <DSXfilename> -
overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO =
DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfile SELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the job last SELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in DataStage..
With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables
used in datastage proj SELECT * FROM SDKSequences;
15List Datastage error message decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI: Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters 1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs <projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams <projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations <projectname> <jobname>
4Generate XML report of job , its stages and links @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -wait -jobstatus -param
<name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop <projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenv
bin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import <projectname> <DSXfilename> -
overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r
Sequencer job -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>
Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop <projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data
Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import <projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)
./uv -admin -stop
./uvregen
./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist
5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows:
ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
DataStage.. With Owner
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>
List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams <projectname> <jobname>
List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations <projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)
unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL
DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>
Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop <projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import <projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using select DS_JOBS.NAME AS JOB_NAME FMT '35L' from
given Hashed file as OUTPUDS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)
./uv -admin -stop
./uvregen
./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist
5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows:
ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
DataStage.. With Owner
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>
List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams <projectname> <jobname>
List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations <projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)
unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL
DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>
Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop <projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import <projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using select DS_JOBS.NAME AS JOB_NAME FMT '35L' from
given Hashed file as OUTPUDS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)
./uv -admin -stop
./uvregen
./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist
5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows:
ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
DataStage.. With Owner
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>
List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams <projectname> <jobname>
List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations <projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)
unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL
DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>
Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop <projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import <projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using select DS_JOBS.NAME AS JOB_NAME FMT '35L' from
given Hashed file as OUTPUDS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)
./uv -admin -stop
./uvregen
./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist
5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows:
ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
DataStage.. With Owner
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>
List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams <projectname> <jobname>
List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations <projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)
unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL
DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>
Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop <projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import <projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using select DS_JOBS.NAME AS JOB_NAME FMT '35L' from
given Hashed file as OUTPUDS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)
./uv -admin -stop
./uvregen
./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist
5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows:
ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
DataStage.. With Owner
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>
List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams <projectname> <jobname>
List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations <projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)
unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL
DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>
Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop <projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import <projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using select DS_JOBS.NAME AS JOB_NAME FMT '35L' from
given Hashed file as OUTPUDS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)
./uv -admin -stop
./uvregen
./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist
5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows:
ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
DataStage.. With Owner
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>
List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams <projectname> <jobname>
List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations <projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)
unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL
DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>
Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop <projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import <projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using select DS_JOBS.NAME AS JOB_NAME FMT '35L' from
given Hashed file as OUTPUDS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)
./uv -admin -stop
./uvregen
./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist
5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in
DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams
<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations
<projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows:
ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
DataStage Administrator commands to find:
No Command Description Command
1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;
SELECT COUNT(*) FROM DS_JOBOBJECTS;
2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS
3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'
CLEAR.FILE RT_Logxx where xx is JOBNO.
4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>
5Find how many Jobs are using given Hashed file as INPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
6Find how many Jobs are using given Hashed file as OUTPU
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
7Find how many Jobs are using given Hashed file as INPUT and OUTPUT
select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;
8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME
9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";
10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');
11Find which user modified the
job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';
12List all the projects in SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;
DataStage.. With Owner
13Reset surrogate key generated using Keymgmt function in Datastage
UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';
14List all the surrogate key variables used in datastage proj
SELECT * FROM SDKSequences;
15List Datastage error message
decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';
16List all jobs using a TABLE
SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;
Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.
Steps to set Datastage Environment, to use Datastage CLI
1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)
1Find total number of connection made from client to Datastage server with the identification of IP address in it.
netstat -a | grep 'dsrpc'
2List all datastage processes executing on the server ps -ef|grep RUN
3Listing projects, jobs, and parameters
1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob
List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs
<projectname>
List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams <projectname> <jobname>
List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations <projectname> <jobname>
4Generate XML report of job , its stages and links
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>
5Do a pattern based search for job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>
List Sequencer jobs calling a specified job in the project
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>
List all jobs called in specified Sequencer job
@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>
6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -
wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop
<projectname> <SeqJobname>
7Start and stop Datastage Server
1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start
8Unlock locked/hung datastage processes
Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)
unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import
<projectname> <DSXfilename> -overwrite
10Troubleshoot Client connectivity from datastage server unix prompt
1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q
Related Documents
