Oracle Database 10g New Features for Developers:
Maximizing the Capabilities of Oracle Database 10g
Daniel SeurerOracle Corporation
AQ&Q U E S T I O N SQ U E S T I O N SA N S W E R SA N S W E R S
Oracle Database 10g Goals
Highest Quality of Service– Highest Availability, Reliability, Security– Highest Performance, Scalability
Easiest to Manage
Agenda
General 10g Database New Features– Recoverabilityb– Business Intelligence– Application Development– Miscellaneous
XML Oracle Provider for .NET HTMLDB
Oracle 10g New FeaturesRecoverability/Security
Flashback Fine Grained Auditing Enhancements
Availability and Recovery:Flashback
Flashback Database– Flashes back changes to entire database– Powerful feature to help in resolution of user
errors that cause changes to database– Much faster method of recovering data than
other methods (tablespace point in time recovery, logical backups, or log miner)
Availability and Recovery:Flashback
Flashback Drop– Allow you to undo the effects of a drop table– Concept of a recycle bin
Flashback Versions Query– Allows you to see all versions of data for a given
row over a period of time
Flashback Transaction– Reconstructs the SQL statements that have been
previously executed in the database
Availability and Recovery:Flashback
Flashback Table– Flashback a table based on a timestamp of
database SCN to the time you are interested in
Fine-grained Auditing
HR.EMP
SELECT NAME, ADDRESS FROM
HR.EMP
SELECT NAME, SALARY FROM HR.EMP WHERE
POSITION = ‘EVP’
Not audited
...AUDIT_CONDITION = ‘salary >
500000’
AUDIT_COLUMN = SALARY
...AUDIT_CONDITION = ‘salary >
500000’
AUDIT_COLUMN = SALARY
Audit Records (FGA_LOG$)
SELECT name,
salary FROM hr.emp
WHERE POSITION =
‘EVP’, <timestamp>,
<userid>, <bind
variables> etc.
SELECT name,
salary FROM hr.emp
WHERE POSITION =
‘EVP’, <timestamp>,
<userid>, <bind
variables> etc.
Audit Policy
SecurityAuditing
Changes to the audit trail view– Type of query executed– Proxy sessionid– Os process
9i only select audited, 10g audits update, insert, delete
Oracle 10g New FeaturesBusiness Intelligence
Oracle’s Business Intelligence Footprint Data Pump Cross Platform Tablespaces Enhanced MERGE functionality Enhanced EXTERNAL TABLE support SQL Model Clause Warehouse Builder OLAP and Data Mining
Metadata
Oracle 10g BI Footprint
OS independent, End-to-End Integrated Business Intelligence Platform
OWB
Reports
Discoverer
Data Mining
BiBeans
End-to-End Metadata
Data Mining
OLAPWarehousing
ETL
Portal
Oracle 10g
DatabaseLegacy
Operational
ERP
Assembly & Integration Access & Analysis Distribution & Sharing
Oracle ETL FeaturesAddressing the entire spectrum
Extract
Change Data Capture
External Tables
SQL*Loader
Data Pump
Transportable Tablespaces
Multi-Table Insert
MERGE
Distributed Queries
Table Functions
Load Transform Insert
Business IntelligenceOracle Data Pump
Replaces traditional imp/exp Imp/Exp still supported
Oracle10g Data Pump
High performance import and export – 60% faster than 9i export (single thread) – 15x-20x faster than 9i import (single thread)
Scales with parallel threads Network transfer data between databases
with no intermediate representation
Business IntelligenceData Pump Export New Features
Ability to estimate files resulting from expdp Suspend/resume expdp at will Ability to do fine-grained object selection
– Thus can choose to only export procedures or functions
Export external tables Support for network node operations
– Load another database without intermediate file
Business IntelligenceData Pump Import New Features
Support use of metadata filters– Control which object types you import
Ability to suspend/resume job at will Support for network node operation Support for direct mode import!!
Business IntelligenceTransportable Tablespaces
Transportable tablespaces can now be moved across platforms
– Windows OS datafiles can be moved to UNIX and vice-versa
Benefits– Efficient publication of data between different
content providers– Easy movement of data between data
warehouses and OLTP systems– Easy migration of databases across platforms
Business IntelligenceExternal Tables
Writing to External tables now supported Project column
– Allows you to determine how access driver will validate rows in table
– You decide whether to have load error out or to continue processing and skipping the invalid column
Business IntelligenceEnhanced MERGE Functionality
Conditional UPDATE and INSERT clauses– Add a where clause to the update or insert operation of a
merge statement– Example (update only occurs if pay_code column is not
equal to the value of RETIRED:
Merge into emp_history eh
Using emp_history_adds eha
On (eh.empno = eha.empno)
WHEN NOT MATCHED THEN
UPDATE SET eh.total_pay = 123.50
WHERE eh.pay_code <> ‘RETIRED’
Change Data Capture
OLTP SystemOLTP SystemTarget SystemTarget System
Online Redo logsOnline Redo logs
Archive logsArchive logs
Mining of archive logs No impact to OLTP system Low Latency Data is exposed as change
tables for further bulk ETL processing via SQL
Publish/Subscribe “Change window” mgmt
Packaging change: OWB will be shipped with the Database as of 10g
New OWB features including– Integrated Data Flow Debugger– Integrated Data Quality features
Name & Address cleansing Complete Match/Merge algorithms
– Enhanced cube generation from general logical design
Oracle Warehouse Builder
Statistical Analysis Descriptive Statistics:
– average, standard deviation, variance, min, max, median (via percentile_count), mode, group-by & roll-up
– DBMS_STAT_FUNCS: summarizes numerical columns of a table and returns count, min, max, range, mean, stats_mode, variance, standard deviation, median, quantile values, +/- 3 sigma values, top/bottom 5 values
Correlations: Pearson’s correlation coefficients, Spearman's and Kendall's (both nonparametric).
Cross Tabs: Enhanced with % statistics: chi squared, phi coefficient, Cramer's V, contingency coefficient, Cohen's kappa
Hypothesis Testing: t-test , F-test, ANOVA, Chi-square, Mann Whitney, Kolmogorov-Smirnov, Wilcoxon signed ranks
Distribution Fitting: normal, uniform, Poisson, exponential, Weibull
New Oracle 10g features are in yellow
Oracle OLAPOracle Business Intelligence Suite
Multidimensional Calculations
Uni-dimensional“What are my top ten customers?”
Multidimensional‘What was the percent change in market share for a grouping of my top 20% of products for the current three month period versus same period year ago for accounts which grew by more than 20 percent in revenue”
Emerging Oracle-Based ApproachInformation and Data Consolidation
Relational & OLAP, databases unite; in-place data mining
Oracle10g
User’s tool of choice
Open standards-based platform for non-Oracle tool users + native Excel support thru add-in
Oracle DiscovererReports, Beans, Portal
Microsoft Excel
Database Managed Integrated OLAP
OLAP in the RDBMS Single RDBMS-MDDS
process Single data storage Single metadata
repository Single set of
management tools Single security model OLAP API and SQL
interfaces
Oracle Enterprise Manager Console
A Look AtRelational
Database TechnologyROLAP
Purely Relational, ROLAP, or MOLAP?
PRODUCT• prod A• prod B• prod C• prod D• prod E SALES
• Prod• Cust• Time
CUST• cust A• cust B• cust C• cust D• cust E
TIME• Year• Quarter• Month• Week• Day
Relational database technology is based on a table/row/column model.
Purely Relational, ROLAP, or MOLAP?Relational Technology
PRODUCT• prod A• prod B• prod C• prod D• prod E SALES
• Prod• Cust• Time
CUST• cust A• cust B• cust C• cust D• cust E
TIME• Year• Quarter• Month• Week• Day
A purely relational implementation is designed and optimized to support the efficient movement and calculation of large volumes of data.
Purely Relational, ROLAP, or MOLAP?Relational Technology
In order to perform multidimensional analysis over a relational data model, a dimensional perspective must be imposed over it.
PRODUCT• prod A• prod B• prod C• prod D• prod E SALES
• Prod• Cust• Time
CUST• cust A• cust B• cust C• cust D• cust E
TIME• Year• Quarter• Month• Week• Day
Cube
Dimension Object
• Dimension• Hierarchy• Level• Attribute
Dimension
Object
Dimension
Object
Dimension
Object
Purely Relational, ROLAP, or MOLAP?Relational Technology
Analytic queries are written in SQL.
PRODUCT• prod A• prod B• prod C• prod D• prod E SALES
• Prod• Cust• Time
CUST• cust A• cust B• cust C• cust D• cust E
TIME• Year• Quarter• Month• Week• Day
Cube
Dimension Object
• Dimension• Hierarchy• Level• Attribute
Dimension
Object
Dimension
Object
Dimension
Object
Purely Relational, ROLAP, or MOLAP?Relational Technology
This is a ROLAP IMPLEMENTATION.
PRODUCT• prod A• prod B• prod C• prod D• prod E SALES
• Prod• Cust• Time
CUST• cust A• cust B• cust C• cust D• cust E
TIME• Year• Quarter• Month• Week• Day
Cube
Dimension Object
• Dimension• Hierarchy• Level• Attribute
Dimension
Object
Dimension
Object
Dimension
Object
Purely Relational, ROLAP, or MOLAP?Relational Technology
A Look AtMultidimensional
Database Technology
Purely Relational, ROLAP, or MOLAP?Multidimensional Technology
SALES(product x, customer y, time z)
The multidimensional database technology that underlies the Analytic Workspace is based on an indexed multidimensional array model.
Purely Relational, ROLAP, or MOLAP?Multidimensional Technology
$ $ $ $
$ $ $ $
$ $ $ $
$ $ $ $
SALES dimensioned by PRODUCT, CUSTOMER, TIME
Cube
This model provides direct cell access, and is designed and optimized for the efficient computation, storage and retrieval of multidimensional data.
Purely Relational, ROLAP, or MOLAP?Multidimensional Technology
$ $ $ $
$ $ $ $
$ $ $ $
$ $ $ $
SALES dimensioned by PRODUCT, CUSTOMER, TIME
Cube
The OLAP DML language, one component of Oracle OLAP, provides an easy way to express complex multidimensional calculations.
Purely Relational, ROLAP, or MOLAP?Multidimensional Technology
$ $ $ $
$ $ $ $
$ $ $ $
$ $ $ $
SALES dimensioned by PRODUCT, CUSTOMER, TIME
Cube
This is a MOLAP implementation
Purely Relational, ROLAP, or MOLAP?Multidimensional Technology
Some benefits of the multidimensional processing model . . .
• No JOIN is required when using the multidimensional
technology. Improved performance!
• “Aggregate then filter” methodology is used. Consistent, correct results. Intelligent drill!.
Purely Relational, ROLAP, or MOLAP?Multidimensional Technology
Select a purely relational implementation when …
• The analytic requirements of the business are met by the capabilities of SQL.
• There are appropriate in-house SQL skills.
• The relational engine provides satisfactory query performance.
Purely Relational, ROLAP, or MOLAP?
Select a ROLAP implementation when …
• The analytic requirements of the business are met by the capabilities of SQL.
• User is looking for an easier way to formulate complex queries.
• The detail data is very sparse.
Use Materialized Views to optimize performance.
Purely Relational, ROLAP, or MOLAP?
Select a MOLAP implementation …
• When the analytic requirements of the business need the extended analytic, forecasting and planning functionality of Multidimensional Database Technology.
• When the analysis includes lots of calculated and aggregated Key Performance Indicators
• Need an easier way to define complex or proprietary calculations.
• Need a transaction model that supports what-if analysis.
Purely Relational, ROLAP, or MOLAP?
Oracle 10G Database
Oracle Data Mining
What is Oracle Data Mining?
Oracle Data Mining (ODM) sifts through massive amounts of data to find hidden patterns and information
— valuable information that can help you better understand your customers and anticipate their behavior
ODM insights can be revealing, significant, and valuable e.g.
– Predict which customers are likely to churn– Discover what factors are involved with a
certain disease– Identify fraudulent behavior
Data Mining Examples Typical problems addressed by data mining include:
What is the probability that a person will…– Buy product X (Marketing)– Stop buying my product and start buying a rival’s product
(Churn)– Default on a loan (Credit Risk)– Submit a false insurance claim (Fraud)– Predict parts failure– Contract a certain type of Leukemia (Life Sciences)
10g Oracle Data MiningWide range of data mining algorithms Feature Selection
– Attribute Importance Supervised learning (classification & prediction)
– Naïve Bayes – Adaptive Bayes Networks– Support Vector Machines
Unsupervised learning (clustering and associations)
– Association Rules – Orthogonal Partitioning Clustering– Enhanced k-means Cluster
Feature Extraction– Non Negative Matrix Factorization
Data Mining
What Is Warehouse Builder?
Warehouse Builder is an open, enterprise level, data and metadata integration tool. It provides a graphical environment for designing, deploying, and managing Business Intelligence applications.
Mining
OLAP
Oracle 10g
Warehousing
ETL Engine
Legacy
Operational
ERP
Oracle Warehouse Builder
Design and Deployment With Oracle Warehouse Builder
Design Target Objects Design Target Objects 1
Map & Transform Map & Transform 3
Generate & Validate Generate & Validate 4
Deploy CodeDeploy Code5
• Relational• Flat Files• Applications• Mainframe
RunRunRunRun66
2 Extract Source Extract Source MetadataMetadata
Application Development
Oracle 10g New FeaturesApplication Development
SQL PL/SQL Regular Expressions Case-insensitive queries New connect string format User defined quote character New PL/SQL Packages of Note PL/SQL Compiler Spatial/Locator
Application DevelopmentRegular Expressions
Very familiar to UNIX developers Set of symbols and elements of syntax that allow you to match
patterns of text within a given expression– Example
* matches zero or more occurences (Ad* matches Ada AdB but not Acd or add
? Matches one character occurrence ^ matches start of line (^abc matches abch but not habc) Example: find all records where ‘b’ is the second character in
the column
Select * from test_expressions
Where regexp_like(char_value,’^.b.’);
Application DevelopmentCase-insensitive queries/sorts
With 10G you can write queries that are case-insensitive WITHOUT using function-based indexes
How? By setting two parameters at either system or session level
– Nls_comp=ansi– Nls_sort=binary_ci
Application DevelopmentCase-insensitive queries/sorts
With 10G you can write queries that are case-insensitive WITHOUT using function-based indexes
How? By setting two parameters at either system or session level
– Nls_comp=ansi– Nls_sort=binary_ci
Application DevelopmentNew Connect String Format
If running on TCP/IP network, Oracle clients like SQL*Plus can connect without tnsnames.ora entry
Just need to supply host, port, and service– Sqlplus test/test@//myhost.oracle.com:1521/prod
Application DevelopmentUser Defined Quote Character
Simplifies writing SQL statements that cotnain literal single quotes in them
– Resulted in really ugly code with many hard to find errors
Delimiter defined by using the quote operator(q), followed by a quote and then the assigned quote delimiter
Application DevelopmentNew Packages of Note
Utl_compress– Compress and decompress BLOB or Raw data
Utl_mail– Send email and attach objects to email
Application DevelopmentPL/SQL Compiler
10g comes with a new, faster PL/SQL compiler
More efficient than previous PL/SQL compiler– 50-75% faster than previous compiler
Resulting native code runs even faster in 10g
Application DevelopmentPL/SQL Compiler
Location-enable Mission Critical Applications
Geocoder Generates latitude/longitude (points) from
address International addressing standardization Formatted and unformatted addresses Tolerance parameters support fuzzy matching 100% Java, open and scalable Record-level and batch processes Data provided by leading data vendors
Server ManageabilitySQL Optimization
Server ManageabilityRule Based Optimizer
Though still there, RBO is desupported with 10g
Packaged& Custom
Applications
Self-Optimizing SQL
Proven Cost-Based Optimizer
CustomizableApplications
Self-Optimizing SQL
Proven Cost-Based Optimizer
Packaged& Custom
Applications
CustomizableApplications
BetterPerformance
High-load SQL
SuggestedIndexes& MVs
Access Advisor
Self-Optimizing SQL
Proven Cost-Based Optimizer
Packaged& Custom
Applications
CustomizableApplications
BetterPerformance
High-load SQL
SuggestedIndexes& MVs
Access Advisor
Self-Optimizing SQL
Proven Cost-Based Optimizer
SQL Advice-> Better
SQL
Auto SQLAnalysis
Packaged& Custom
Applications
CustomizableApplications
BetterPerformance
High-load SQL
SuggestedIndexes& MVs
Access Advisor
Self-Optimizing SQL
Proven Cost-Based Optimizer
SQL Profile-> Improved
Plan
SQL Advice-> Better
SQL
Auto SQLAnalysis
Auto SQL Tuning
Packaged& Custom
Applications
CustomizableApplications
BetterPerformance
High-load SQL
SuggestedIndexes& MVs
Access Advisor
BREAK
Developing Oracle Application with .NET
Why Oracle for .Net? Oracle brings enterprise scalability, availability and
reliability to .NET– Advanced DB features are available through Oracle
Data Provider for .NET (ODP.NET)
Oracle is open– DB server can be on Windows, Linux, or Unix– Developers can use either .NET or J2EE without losing
DB capabilities
Wide adoption– 50k+ ODP.NET downloads in past year
Develop with .NetDatabase on any platform
Visual StudioEnvironment
Oracle Developer Oracle Developer Tools for VS.NETTools for VS.NET
Windows
Windows
Linux
Unix
Oracle 10g, 9i, or 8i
Other
Oracle’s Commitment to .NET
Mid 200520032002
.NET Framework
1.1
.NET Framework
1.0
1H 2005
ODP.NET
Oracle Developer Tools for VS.NET
ODP.NET
Oracle Developer Tools for VS.NET
Oracle Database Extensions for .NET
Oracle Data Provider for .NET(ODP.NET)
.NET Framework
2.0
ODP.NET fully compliant with ADO.NET Oracle DB leads the way in advanced functionality ---
all exposed in ODP.NET– RAC, XML DB, Advanced Security, etc.
Support for Native Oracle data types– LOBs, REF Cursor, N-data types, Date,
TimeStamps, Long, etc. Full PL/SQL support
– Batch SQL available with anonymous PL/SQL
Oracle Data Provider for .NET
Available today in production for free– http://otn.oracle.com/tech/windows/odpnet– Original release: 2002
Can be used with Oracle8i, Oracle9i, and Oracle10g database servers
– DB server can be on Unix, Linux, Windows, etc.– Database client on Windows
Supports VS.NET 2002 and 2003 Supports .NET Framework 1.0 and 1.1
Oracle Data Provider for .NET
Oracle Developer Tools for Visual Studio .NET
New tightly integrated toolset for VS .NET Stay in VS .NET environment for project
lifecycle Powerful features that enhance productivity –
e.g. designers, code generation Exposes native Oracle functionality not
typically available in generic tools Easy to learn and intuitive
ODP.NET Summary
Oracle brings the power of Oracle Database to .Net Developers
Oracle is committed to delivering the best database for Windows and making Windows developers more productive when using Oracle Database
Oracle has best tools for whichever development platform you choose
Sample Code, Tutorials & More Information
.Net Development Center– http://otn.oracle.com/dotnet
Oracle XML DB
Common XML Architectures
Enforce XML Schema
Manage Content & DataUniformly
Eliminate extra movingparts
Get better performance& scalability
Make DB applicationsStandards-based
Evolution of Oracle XML Support
8i 9i R1 9i R2
Basic XML Generation andProcessing (mid-tier based)Developer Kits
DB-integrated XML StorageAnd Retrieval
Native XML DB
What is Oracle XML DB? Infrastructure & set of features to store and manage XML data Allows DB to absorb W3C XML Data Model Storage Options:
– Unstructured Storage: Documents are stored as CLOB– Structured Storage: XML documents are shredded into a set
of SQL objects in a table based on SQL99 object standard Major Components of XML DB
– XMLType– Access via SQL, PL/SQL, OCI, JDBC, SQLJ – DOM APIs in PL/SQL, C, and Java– Java Clients via a Bean interface
– XML Schema supports W3C XML Schema Standard– Support for XPATH expressions to query/update XML
content– XML DB Repository– SQL/XML
Oracle 10g New FeaturesHTMLDB
Does Your Organization:
Use spreadsheets as databases? Store mission critical data in applications
developed in a desktop database such as Microsoft Access?
Not have time to build all of the applications that users are asking for?
Drawbacks to Desktop DBs
VULNERABLEData not Secure
COSTLYInefficient use of IT Resources
FRAGMENTEDData in Many Places
PLATFORM DEPENDENTMS Windows Only
WEB UNFRIENDLYNeed to Install Client Software
Access Password “Recovery”
About $15 Many variations Also available for
other MS Office Components
If your data is worth less than $15, MS Access is a good place to store it
Consolidation with HTML DB
Spreadsheets
Desktop Databases
File Servers Any Modern Browser
Oracle Database & Oracle HTML DB
The Best of Both Worlds The Qualities of a Personal Database
– Productivity– Ease of Use– Flexibility
The Qualities of an Enterprise Database– Security– Integrity– Scalability– Availability– Portability
Built for the Web
Physical Architecture
BrowserOracle Database
HTML DBOracle HTTP
Server
Physical Architecture
BrowserOracle Database
HTML DBOracle HTTP
Server
Firewall
Instead of the Database Oracle HTTP Server, you can use the
HTTP Server supplied with Oracle Application Server 9i or 10g
HTML DB Architecture
Oracle Database
HTML DBEngine
Application MetadataBrowserOracle Database
HTML DBOracle HTTP
Server
Components
Administration– Manage Workspaces & System Settings
Data Workshop– Get data in & out of the Database
SQL Workshop– Manage your schema objects
Application Builder– Build and deploy web applications
Integration
Oracle Database 10g
– RAC– Label Security– Virtual Private
Database– Regular Expressions– Recycle Bin– Spatial– Oracle Text– Flashback Query
Oracle App Server 10g
– MapViewer– Oracle Internet
Directory Authentication Groups
– Oracle Login Server Partner Application External Application
– Oracle Portal OmniPortlet
Integration
Oracle Database 10g
– RAC– Label Security– Virtual Private
Database– Regular Expressions– Recycle Bin– Spatial– Oracle Text– Flashback Query
Oracle App Server 10g
– MapViewer– Oracle Internet
Directory Authentication Groups
– Oracle Login Server Partner Application External Application
– Oracle Portal OmniPortlet
Availability
Release 1.5– Software ships on Oracle Database 10g Companion CD
Release 1.6– Available as a Download from OTN
Works & Supported in Oracle Database 10g and Oracle Database 9iR2 (9.2.0.3+)
Included as a component of Standard One, Standard and Enterprise Editions
No Additional Charge
AQ&Q U E S T I O N SQ U E S T I O N SA N S W E R SA N S W E R S