8/22/2019 B301 AU Intro
1/15
Module 1: Introduction to Application Utilities
After completing this module, you should be able to:
Identify the Application Utilities.
Describe how the Application Utilities interface withthe Teradata database.
State the advantage of using a utility over other
access methods.
Match Teradata Warehouse Builder operators with thecorresponding Teradata utility.
8/22/2019 B301 AU Intro
2/15
Application Utilities
Teradata
Database
Host or Server
Operating System
MultiLoad
Support
Environment
CLI
Routines
FastLoad
CLI
Routines
BTEQ /Teradata SQL
CLI
Routines
FastExport
Support
Environment
CLI
Routines
TPump
Support
Environment
CLI
Routines
Teradata
Warehouse
Builder
(operators)
CLI
Routines
8/22/2019 B301 AU Intro
3/15
Application Utility Environments
BTEQFastLoad
MultiLoad
FastExport
TPump
Call Level Interface
Micro TDP
Server O.S.
TCP / IP
NCR Server
BTEQFastLoad
MultiLoad
FastExport
TPump
Call Level Interface
TDP 0
Host O.S.
TDP 1
Mainframe Host
AMP
0 AMP1 AMP5AMP3 AMP4AMP2 AMP6 AMP7
U
N
I
X
P
D
EBYNET
PE PE
Gateway Channel Driver
8/22/2019 B301 AU Intro
4/15
Application Development
Flexibility
Ease of Use DifficultEasy
Less
More
Call-Level
Interface
COBOL, C, PL/I
PreprocessorsAPPLICATION
UTILITIES
Input Modification
Routines(INMODs)
Selection of the right vehicle can be crucial to the success of the application:
How difficult is it to implement?
How difficult is it to maintain?
Use the application utilities wherever possible:
Offer the least complexity. Take full advantage of parallel processing.
8/22/2019 B301 AU Intro
5/15
Transferring Large Amounts of Data
TDP/Gateway
BTEQImport/Export
Both Directions Teradata to HOST
FastLoad
HOST to Teradata
MultiLoad TPump FastExport
Message Passing Layer
Teradata Database Parsing Engine Parsing Engine
SQL
INSERT/SELECT
DELETE ALL
Block
SQL
INSERT/SELECT
DELETE ALL
Block
SQL
INSERT/SELECT
DELETE ALL
Block
SQL
INSERT/SELECT
DELETE ALL
Block
Teradata
Warehouse
Builder
Both Directions
Host or Server
8/22/2019 B301 AU Intro
6/15
INSERT/SELECT: The Fast Path
INSERT INTO New_Table
SELECT * FROM Old_Table ;
Message Passing Layer
Parsing Engine Parsing Engine
SQL
INSERT/SELECT
Block
SQL
INSERT/SELECT
Block
SQL
INSERT/SELECT
Block
SQL
INSERT/SELECT
Block
INSERT / SELECT achieves highest performance if:
Target table is empty, AND
Source and target tables have same Primary Index.
Advantages of using optimized INSERT / SELECT:
One WRITE to the Transient Journal instantaneous rollback for aborted
Insert/Select statements.
Data copied and written to disk a block at a time.
No data redistribution over the BYNET.
8/22/2019 B301 AU Intro
7/15
Multi-Statement INSERT/SELECT Example
SPOOL
Optimized INSERT/SELECT
Region_1 Region_2 Region_N
Empty Target Table
INSERT INTO Summary_Table
SELECT store, region,
SUM(sales),
COUNT(sale_item)
FROM Region_1
GROUP BY 1, 2
; INSERT INTO Summary_Table
SELECT store, region,
SUM(sales),
COUNT(sale_item)
FROM Region_2
GROUP BY 1, 2
. . .
; INSERT INTO Summary_Table
SELECT store, region,
SUM(sales),
COUNT(sale_item)
FROM Region_N
GROUP BY 1, 2
;
8/22/2019 B301 AU Intro
8/15
DELETE (ALL): The Fast Path
DELETE FROM Old_Table ALL;
Message Passing Layer
Parsing Engine Parsing Engine
DELETE achieves its highest performance with use of the ALL option.
High performance is achievable because of:
Not using the Transient Journal.
Doing DELETEs at the cylinder index / master index level.
SQL
DELETE ALL
Block Free
SQL
DELETE ALL
Block Free
SQL
DELETE ALL
Block Free
SQL
DELETE ALL
Block Free
8/22/2019 B301 AU Intro
9/15
Using an INMOD Routine
An INMOD is a user-written program that acts as a data filter to the utility.
An INMOD routine can perform various functions:
Validate a data record.
Read data directly from one or more database system data sets, allowing
the creation of a composite input data record, and avoiding the need for an
intermediate tape or disk.
Select specific records for input to the Teradata Database.
Perform data conversions not supported by the application utilities.
Add or change data fields in the records.
TeradataOracle
IMSFASTLOAD, MULTILOAD,
TPUMP. . .User Written
INMOD
An OUTMOD is a user-written program that does post-processing of the databefore giving the data to the the utility.
8/22/2019 B301 AU Intro
10/15
Teradata Warehouse Builder
Teradata Warehouse Builder (TWB) can load data into and export data from any
accessible database object in the Teradata RDBMS or other data store for whichthere exists an access operator or an access module.
It is effectively a parallel ETL utility. Characteristics of TWB include:
High performance
Parallel Extract, Transform and Load operators eliminate sequentialbottlenecks.
Data Streams eliminate the overhead of intermediate (persistent) storage.
Scalable
End-to-end parallelism
Easy to use Single SQL-like scripting language
Script converter for easy migration
Access to various data sources
Extensible
Open TWB API enable Third Party and user application integration, e.g.,transformations.
8/22/2019 B301 AU Intro
11/15
Teradata Warehouse Builder Operators
To learn more about Teradata Warehouse Builder, take the Teradata Warehouse BuilderWBT Course located at ... http://www.ncru.learnzone.ncr.com/learning/td/twb/
TWB Operator Teradata Utility Description
LOAD FastLoad A consumer-type operator that uses the TeradataFastLoad protocol. Supports Error limits and
Checkpoint/ Restart. Both support Multi-Value
Compression and PPI.
UPDATE MultiLoad Utilizes the Teradata MultiLoad protocol to enable job
based table updates. This allows highly scalable andparallel inserts and updates to a pre-existing table.
EXPORT FastExport A producer operator that emulates the FastExportutility.
STREAM TPump Uses multiple sessions to perform DML transactionsin near real-time.
DataConnector N/A This operator emulates the Data Connector API. Readsexternal data files, writes data to external data files,
reads an unspecified number of data files.
ODBC N/A Reads data from an ODBC Provider.
8/22/2019 B301 AU Intro
12/15
Application Utility Checklist
Feature BTEQ FastLoad FastExport MultiLoad TPump
DDL Functions
DML Functions
Multiple DML
Multiple Tables
Multiple Sessions
Protocol Used
Conditional Expressions
Arithmetic CalculationsData Conversion
Error Files
Error Limits
User-written Routines
8/22/2019 B301 AU Intro
13/15
Application Utility Summary
BTEQ supports SELECT, INSERT, UPDATE, DELETE.
FastLoad, MultiLoad, and TPump transfer data from the host to Teradata.
FastExport transfers data from Teradata to the host.
BTEQ INSERT/SELECT and DELETE (ALL) can provide a fast effective
method to perform some tasks.
Utilities offer the least complex solutions for an application, and can take
advantage of parallel processing.
Utilities permit the use of INMODs and/or OUTMODs for pre- or post-
processing data.
There is often more than one way to set up your application, but there may be
one that is fastest or most effective.
Teradata Warehouse Builder (TWB) can load data into and export data from
any accessible database object in the Teradata RDBMS or other data store for
which there exists an access operator.
8/22/2019 B301 AU Intro
14/15
Review Questions
Answer True or False.
1. True or False. With MultiLoad, you can import and export data.
2. True or False. In Teradata mode, a BTEQ DELETE ALL function does not use the Transient
Journal.
3. True or False. An INSERT/SELECT of 1,000,000 million rows into an empty table is only slightly
faster than an INSERT/SELECT of 1,000,000 rows into a table with 1 row.
Match the Teradata Warehouse Builder operator with the corresponding Teradata utility.
1. ___ UPDATE A. MultiLoad
2. ___ STREAM B. FastLoad
3. ___ LOAD C. FastExport
4. ___ EXPORT D. TPump
8/22/2019 B301 AU Intro
15/15
Module 1: Review Question Answers
Answer True or False.
1. True orFalse. With MultiLoad, you can import and export data.
2. Trueor False. In Teradata mode, a BTEQ DELETE ALL function does not use the Transient
Journal.
3. True orFalse. An INSERT/SELECT of 1,000,000 million rows into an empty table is only slightly
faster than an INSERT/SELECT of 1,000,000 rows into a table with 1 row.
Match the Teradata Warehouse Builder operator with the corresponding Teradata utility.
1. _A_ UPDATE A. MultiLoad
2. _D_ STREAM B. FastLoad
3. _B_ LOAD C. FastExport
4. _C_ EXPORT D. TPump