SQLLOADER

What Is It?Oracle utility which populates Oracle tables from host files

Tables must be created firstSQL*LOADER operates with control fileData can be put into control file or stored

in separate data file.High Performance Data tool.Data can loaded from any text file and

inserted into the DB

1

SQL Loader ArchitectureDuring processing, SQL*Loader writes

messages to the log file, bad rows to the bad file, and discarded rows to the discard file.

Marina G. Erechtchoukova 2

Data Positioned file

Each row starts from the most left position.Each field takes the number of positions

according to its domain constraintFiles with delimiter

Each record is placed into one rowEach field is delimited by a symbol

3

Control fileSpecifies the action:

InsertReplaceAppend

Specifies the data file name and list of fieldsHas an extension “.ctl”

4

Control File for Positioned Data FileLOAD DATA

INFILE File_name REPLACE INTO TABLE Table_name

(field_1 POSITION (SP:EP) data_type_1,…,

field_K POSITION (SP:EP) data_type_K) )

5

Control File for Data File with DelimiterLOAD DATA

INFILE File_name REPLACE INTO TABLE Table_name

FIELDS TERMINATED BY ‘,’ OPTIONALLY ENCLOSED BY “’”(field_1, field_2, …, field_K)

6

Control File Contains Data Control file is prepared in the same wayINFILE *BEGINDATA clause is addedBelow the clause data are placed according

to the format declared in the control information

7

Using the SQL*LOADERCreate tablePrepare Control file and data file or place

data into control fileInvoke the utility without arguments to get

a list of available parameters . ex: sqlldr

username@server/password control=loader.ctl sqlldr username/password@server control=loader.ct

8

The loader.ctl loads an external data file con delimiter data.

Can also work with tabulation. Ex: fields terminated by "\t" fields

terminated by X'09

9

How Does One Load MS-Excel Data Into OracleOpen the MS-Excel spreadsheet and save it

as a CSV (Comma Separated Values) file. This file can now be copied to the Oracle machine and loaded using the SQL*Loader utility.

There can arise some possible errors since the spread sheet may contain new line characters ,where as SQL Loader expects it to be in a single line.

To remove the new line character(Tools -> Macro -> Visual Basic Editor)

10

Using the SQL*LOADER in ITEC Lab Login onto sit.yorku.ca Call SQL*LOADER utility

sqlldr login@studb10g Control_File System asks for your password Enter your password

11

If You Get an Error…Control file has an extension other than “.ctl”You did not create the table firstYou use different column names in the

control file and in the table you createdSQL*LOADER creates “.log” fileSQL*LOADER creates “.bad” file if an error

occurs.

12

Example: Control FileLOAD DATAINFILE 'site.dat' REPLACE INTO TABLE Site FIELDS TERMINATED By ',' OPTIONALLY

ENCLOSED BY "'" (site_id, location)

13

Example: Data File1, 'Paris'2, 'Boston'3, 'London‘4, ‘Ottawa’5, ‘Toronto’

14

Example: Data in the Control FileLOAD DATAINFILE * REPLACE INTO TABLE building3FIELDS TERMINATED BY '|'( B_NAME, HF, DID, B_SIZE)BEGINDATAgreen|y|d3|20red|y|d1|18blue|n|d2|16

15

Methods of downoading data to a flat file

This can be done using SQL*PLUS or Pl/Sql

Ex below using Sql*PLUS & PL/Sql respectively:

set echo off newpage 0 space 0 pagesize 0 feed off head off trimspool on spool oradata.txt select col1 || ',' || col2 || ',' || col3 from tab1 where col2 = 'XYZ'; spool off.

declare fp utl_file.file_type; begin fp := utl_file.fopen('c:\oradata','tab1.txt','w'); utl_file.putf(fp, '%s, %sn', 'TextField', 55); utl_file.fclose(fp); end;

16

Data Can be modified as its been loaded.Column can also be populated with derived and

static values.Data can also be loaded from multiple files into

multiple table all at once Ex :LOAD DATA INFILE file1.dat INFILE file2.dat INFILE

file3.dat APPEND INTO TABLE emp ( empno POSITION(1:4) INTEGER EXTERNAL, ename POSITION(6:15) CHAR, deptno POSITION(17:18) CHAR, mgr POSITION(20:23) INTEGER EXTERNAL )

Ex: loading into multiple tables LOAD DATA INFILE * INTO TABLE tab1 WHEN tab =

'tab1' ( tab FILLER CHAR(4), col1 INTEGER ) INTO TABLE tab2 WHEN tab = 'tab2' ( tab FILLER POSITION(1:4), col1 INTEGER ) BEGINDATA tab1|1 tab1|2 tab2|2 tab3|3

17

FILLER columns are used to skip columns/fields in the load file, ignoring fields that one does not want.

One can create one logical record from multiple physical records using one of the following two clauses:

CONCATENATE - use when SQL*Loader should combine the same number of physical records together to form one logical record.

CONTINUEIF - use if a condition indicates that multiple records should be treated as one. Eg. by having a '#' character in column 1.

18

Methods of Improving the performance of a sql loaderA very simple but easily overlooked hint is

not to have any indexes and/or constraints (primary key) on your load tables during the load process. This will significantly slow down load times even with ROWS= set to a high value.

Add the following option in the command line: DIRECT=TRUE. This will effectively bypass most of the RDBMS processing.

19

Contd…However, there are cases when you can't use

direct load. For details, refer to the FAQ about the differences between the conventional and direct path loader below.

Turn off database logging by specifying the UNRECOVERABLE option. This option can only be used with direct data loads.

Run multiple load jobs concurrently.

20

21