Dale Roberts 1 Department of Computer and Information Science, School of Science, IUPUI Dale Roberts, Lecturer Computer Science, IUPUI E-mail: [email protected] SQL Utilities SQL Utilities
Dec 15, 2015
Dale Roberts 1
Department of Computer and Information Science,School of Science, IUPUI
Dale Roberts, Lecturer
Computer Science, IUPUI
E-mail: [email protected]
SQL UtilitiesSQL Utilities
Dale Roberts 2
SQL UtilitiesSQL Utilities
1.1. SQL*LoaderSQL*Loader
2.2. Data Pump Export and ImportData Pump Export and Import
3.3. Accessing Remote Data (Database Links, Oracle Accessing Remote Data (Database Links, Oracle Net)Net)
4.4. Materialized Views (aka Snapshots)Materialized Views (aka Snapshots)
5.5. Data Dictionary ViewsData Dictionary Views
Dale Roberts 3
SQL*Loader – Bulk LoadSQL*Loader – Bulk LoadSQL*Loader, executed as sqlldr, loads data from external files into tables in SQL*Loader, executed as sqlldr, loads data from external files into tables in Oracle.Oracle.
Very common utility used to process external input interfaces from other Very common utility used to process external input interfaces from other applications.applications.
Uses a control file to describe the input data files.Uses a control file to describe the input data files.
Control file is flexible and supports the following features:Control file is flexible and supports the following features:
Variable and fixed length record formatsVariable and fixed length record formats
Many-to-one physical to logical recordsMany-to-one physical to logical records
One-to-many physical to logical recordsOne-to-many physical to logical records
Data transformationsData transformations
NULLIFNULLIF
Functions – UPPER(:Authorname)Functions – UPPER(:Authorname)
Expressions - :taxamount / 100Expressions - :taxamount / 100
Formatting – DATE(:birthdate, ‘MM-DD-YYYY’)Formatting – DATE(:birthdate, ‘MM-DD-YYYY’)
Default column values – SYSDATEDefault column values – SYSDATE
Row selection - WHENRow selection - WHEN
Batch and interactive interfacesBatch and interactive interfaces
Supports partial loads set error and discard tolerance levelsSupports partial loads set error and discard tolerance levels
Dale Roberts
SQL*Loader ArchitectureSQL*Loader ArchitectureSQL*Loader runs locally on the SQL*Loader runs locally on the client, therefore all files resides on client, therefore all files resides on the client except for the database the client except for the database tables.tables.
Input files contain data to be Input files contain data to be loaded.loaded.
Control file contains instructions Control file contains instructions to sqlldr, including describing the to sqlldr, including describing the input files.input files.
Log file contains a detailed Log file contains a detailed summary of the load, including a summary of the load, including a description of any errors that description of any errors that occurred. Can set max allowed.occurred. Can set max allowed.
Discard file contains rows that are Discard file contains rows that are filtered out by the control file and filtered out by the control file and did not make it to the database. did not make it to the database. Can set max allowed.Can set max allowed.
4
Dale Roberts 5
SQL*Loader Control File ExamplesSQL*Loader Control File ExamplesVariable-length field control fileVariable-length field control file
Fixed-length field control fileFixed-length field control file
Dale Roberts
Data Pump OverviewData Pump Overview
Oracle Data Pump technology enables very high-speed Oracle Data Pump technology enables very high-speed movement of data and metadata from one database to movement of data and metadata from one database to another.another.
Data Pump Export is a utility for unloading data and Data Pump Export is a utility for unloading data and metadata into a set of operating system files called a metadata into a set of operating system files called a dump file set. dump file set.
The dump file set can be moved to another system and The dump file set can be moved to another system and loaded by the Data Pump Import utility.loaded by the Data Pump Import utility.
Data Pump Import is a utility for loading an export dump Data Pump Import is a utility for loading an export dump file set into a target system. The dump file set is made file set into a target system. The dump file set is made up of one or more disk files that contain table data, up of one or more disk files that contain table data, database object metadata, and control information. database object metadata, and control information.
The files are written in a proprietary, binary format.The files are written in a proprietary, binary format.
6
Dale Roberts 7
Data Pump Export OperationData Pump Export Operation
Network
Export File(s)
expdp.exe
Oracle Database
Dale Roberts
Types of ExportsTypes of Exports
TableTable
SchemaSchema
TablespaceTablespace
DatabaseDatabase
Transportable TablespaceTransportable Tablespace
INCLUDE / EXCLUDE object filtersINCLUDE / EXCLUDE object filters
QUERY and SAMPLE data filtersQUERY and SAMPLE data filters
CONTENTS = data | metadata | bothCONTENTS = data | metadata | both
Dale Roberts
Directory ObjectsDirectory Objects
Created as a database objectCreated as a database object
Requires Requires CREATE_ANY_DIRECTORYCREATE_ANY_DIRECTORY privilege privilege
Permissions (read, write) granted on the Permissions (read, write) granted on the object to specific user(s)object to specific user(s)
Not validated – existence, syntax, OS Not validated – existence, syntax, OS privilegeprivilege
Accessed as user “oracle” at the OS levelAccessed as user “oracle” at the OS level
Default Default DATA_PUMP_DIR DATA_PUMP_DIR maps tomaps to … …
Dale Roberts
Data Punp Interactive ModeData Punp Interactive Mode
NOT the same as old imp/exp!NOT the same as old imp/exp!
Default starts schema mode exportDefault starts schema mode export
Use command line arguments or par fileUse command line arguments or par file
““logging” logging” vsvs “interactive command” mode “interactive command” mode
Default logging mode logs to terminalDefault logging mode logs to terminal
Ctrl-C to enter interactive command modeCtrl-C to enter interactive command mode
Job will continue to run even if client Job will continue to run even if client disconnects!disconnects!
expdp scott/tiger@fred parfile=myjob.parexpdp scott/tiger@fred parfile=myjob.par
Dale Roberts
Interactive Mode CommandsInteractive Mode Commands
STATUS Status of current job
CONTINUE_CLIENT Resume “logging” mode
EXIT_CLIENT Exit client, leave job running
STOP_JOB Stop current job, do not delete
START_JOB Re-start current job
KILL_JOB Delete current job
PARALLEL Add/remove worker processes
ADD_FILE Add dump file during export
HELP Get help
Dale Roberts
Data Pump SQL FileData Pump SQL File
Import can generate an SQL file instead of Import can generate an SQL file instead of actually performing the import using SQLFILE actually performing the import using SQLFILE parameterparameter
Contains DDL that would have been executed Contains DDL that would have been executed based on job parametersbased on job parameters
Passwords excludedPasswords excluded
No change to target DBNo change to target DB
Does not include DML for data. (i.e. no inserts)Does not include DML for data. (i.e. no inserts)
Dale Roberts 13
Database LinksDatabase Links
Database links allow you to access data in another Oracle database.Database links allow you to access data in another Oracle database.
Requires that Oracle Net (aka Oracle SQL*Net or Net8) be running on both Requires that Oracle Net (aka Oracle SQL*Net or Net8) be running on both database servers.database servers.
Links specify:Links specify:Protocol (eg. TCP/IP)Protocol (eg. TCP/IP)
Host name of remote server.Host name of remote server.
Database name on the remote serverDatabase name on the remote server
The database account and password to access the data.The database account and password to access the data.
Database links expressed with @ sign: practice.ledger@firebirdDatabase links expressed with @ sign: practice.ledger@firebird
Dale Roberts 14
Database LinksDatabase Links
Create Database Link syntax:Create Database Link syntax:CREATE DATABASE LINK MYLINKCREATE DATABASE LINK MYLINKCONNECT TO DALE IDENTIFIED BY PASSWORDCONNECT TO DALE IDENTIFIED BY PASSWORDUSING ‘FIREBIRD’;USING ‘FIREBIRD’;
Using a database link:Using a database link:SELECT COUNT(*) FROM DALE.MYTABLE@MYLINK;SELECT COUNT(*) FROM DALE.MYTABLE@MYLINK;
Using a synonym with a database link (location independence):Using a synonym with a database link (location independence):CREATE SYNONYM MYTABLE FOR DALE.MYTABLE@MYLINK;CREATE SYNONYM MYTABLE FOR DALE.MYTABLE@MYLINK;SELECT COUNT(*) FROM MYTABLE;SELECT COUNT(*) FROM MYTABLE;
““Dynamic” links using SQL*Plus’s copy command:Dynamic” links using SQL*Plus’s copy command:COPY FROM DALE/PASSWORD@MYLINKCOPY FROM DALE/PASSWORD@MYLINKCREATE NEWTABLECOPY CREATE NEWTABLECOPY USING USING SELECT * FROM MYTABLE;SELECT * FROM MYTABLE;
Dale Roberts 15
Oracle NetOracle Net Oracle Net is Oracle’s networking software. Formerly called SQL*Net (7.x) and
Net8 (8.0.x and 8i). Oracle Net’s local configuration files are sqlnet.ora and tnsnames.ora. These
are located in $ORACLE_HOME\network\admin. Example tnsnames.ora entries below.
LOCAL = (DESCRIPTION = (SOURCE_ROUTE = OFF) (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = DALE)(PORT = 1521)) ) (CONNECT_DATA = (SERVICE_NAME = ORACLE) ) )FIREBIRD = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = firebird.cs.iupui.edu)(PORT = 1521)) ) (CONNECT_DATA = (SERVICE_NAME = cs10gorc.cs.iupui.edu) ) )
Dale Roberts 16
Materialized ViewsMaterialized Views
Materialized Views (formerly called Snapshots) pre-aggregate data, index the data, and therefore improve query performance. Oracle creates a table that holds the data that otherwise might be accessed from queries/views.
Materialized views are commonly used in reporting applications.• Why materialized views instead of views?• Classic space vs time trade-off.
When you create a materialized view, you must specify:• The query the materialized view is based on.• The refresh schedule.• How the update is performed (update vs. complete refresh)• The key type (rowid vs. primary key)
Dale Roberts 17
Data Dictionary - MetadataData Dictionary - Metadata
Oracle’s Data Dictionary views stores all the information about what is stored in the database.
Three types of data dictionary views:• DBA_% views are only available to DBAs (granted DBA role) .• ALL_% views show all objects the user has privileges on.• USER_% views show only objects in your schema.
The Road Map views “meta metadata data”: DICT and DICT_COLUMNS• DICTIONARY (DICT) – describes other dictionary views• DICT_COLUMNS – describes columns of DICT views
Data Dictionary views can be found in the SYS tablespace but have public synonyms so no schema owner is necessary.
Access to production source code:• Viewing production source code requires compilation privilege,
which is not normally allowed in production.• Granting select access to DBA_SOURCE is one workaround.
Dale Roberts 18
Data DictionaryData Dictionary
Important Data Dictionary views:• USER_TABLES (TABS) – tables• USER_TAB_COLUMNS (COLS) – columns of tables• USER_VIEWS – views• USER_SYNONYMS (SYN) – synonyms• USER_SEQUENCES (SEQ) – sequences• USER_CONSTRAINTS – constraints• USER_CONS_COLUMNS – columns of the constraints• USER_INDEXES (IND) – indexes• USER_IND_COLUMNS – columns of indexes
Dale Roberts 19
AcknowledgementsAcknowledgements
Loney, Oracle Database 10g The Complete ReferenceLoney, Oracle Database 10g The Complete Reference
Calgary Oracle User‘s Group, Calgary Oracle User‘s Group, www.coug.ab.ca..