#C14LV REMINDER Check in on the COLLABORATE mobile app #C14LV Something Old, Something New: Leveraging Oracle 12c’s Information Lifecycle Management (ILM) Features for Improved Database Performance Session #187 Jim Czuprynski Zero Defect Computing, Inc. April 9, 2014
26
Embed
Jim Czuprynski Zero Defect Computing, Inc. April 9 , 2014
Something Old, Something New: Leveraging Oracle 12c’s Information Lifecycle Management (ILM) Features for Improved Database Performance Session #187. Jim Czuprynski Zero Defect Computing, Inc. April 9 , 2014. My Credentials. 30+ years of database-centric IT experience Oracle DBA since 2001 - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
#C
14
LV
REMINDER
Check in on the COLLABORATE mobile app
#C
14
LV
Something Old, Something New: Leveraging Oracle 12c’s Information Lifecycle Management (ILM) Features for Improved Database PerformanceSession #187
Jim CzuprynskiZero Defect Computing, Inc.April 9, 2014
#C
14
LV
My Credentials
■ 30+ years of database-centric IT experience
■ Oracle DBA since 2001
■ Oracle 9i, 10g, 11g OCP and Oracle ACE Director
■ > 100 articles on databasejournal.com and ioug.org
■ Regular speaker at Oracle OpenWorld, IOUG COLLABORATE, OUG Norway, and Hotsos
■ Oracle-centric blog (Generally, It Depends)
#C
14
LV
Our Agenda
■The Persistence of Data Through Time■Information Lifecycle Management (ILM)■In-Database Archiving (IDA) Concepts■Temporal Validity (TV) Concepts■Temporal History: Old Feature, New
Name■Leveraging ILM ADO Features■Q+A
#C
14
LV
The Persistence of Data Through Time
■Data needs to be retained for inordinately long periods of time▪ Data mining / warehousing / trend analysis▪ Regulatory and legal requirements
■Once data has been purged … it’s often needed again (usually immediately)
■Therefore, data purging doesn’t happen nearly as often as it should
■Handling schema changes after archiving is a nightmare …
■… so, what if we never purged data at all?
#C
14
LVInformation Lifecycle Management (ILM)
ILM encompasses three key feature sets:■In-Database Archiving (IDA) stores active and
inactive data within same table■Temporal Validity (TV) stores past, current,
and future data within same table■Automatic Data Optimization (ADO) tiers data
according to its age and usage as well as how much space it consumes
#C
14
LV
In-Database Archiving (IDA): A Cloaking Device
#C
14
LV
In-Database Archiving (IDA)
■Avoids unnecessary deletion of rows when they no longer contain valid data
■Activated via new ROW ARCHIVAL attribute of data segment▪ During initial INSERT, each row’s state is set to active by
placing a zero (0) in the ORA_ARCHIVE_STATE hidden column
▪ A row can be marked as inactive by setting ORA_ARCHIVE_STATE to any non-zero value
■Unless the ORA_ ARCHIVE_STATE column is mentioned in query, a row’s IDA status is invisible
■Inactive rows can be compressed via ILM
#C
14
LV
Activating IDA
CREATE table oe.currency_conversions ( currency_from VARCHAR2(3) NOT NULL ,currency_to VARCHAR2(3) NOT NULL ,effect_dtm TIMESTAMP NOT NULL ,conversion_fctr NUMBER(10,6) DEFAULT 0) TABLESPACE example;
ALTER TABLE oe.currency_conversions ROW ARCHIVAL;
Activates In-Database
Archiving for an existing table
CREATE table oe.currency_conversions ( currency_from VARCHAR2(3) NOT NULL ,currency_to VARCHAR2(3) NOT NULL ,effect_dtm TIMESTAMP NOT NULL ,conversion_fctr NUMBER(10,6) DEFAULT 0) ROW ARCHIVAL TABLESPACE example;
Activates In-Database Archiving
immediately upon table
creation
#C
14
LV
COL column_name FORMAT A30 HEADING "Column Name"COL segment_column_id FORMAT 9999 HEADING "Seg|Col|#"COL internal_column_id FORMAT 9999 HEADING "Int|Col|#"COL hidden_column FORMAT A08 HEADING "Hidden?"COL virtual_column FORMAT A08 HEADING "Virtual?"COL user_generated FORMAT A08 HEADING "User|Gen'd?"TTITLE "Metadata for Table OE.CURRENCY_CONVERSIONS|(from DBA_TAB_COLS)"SELECT column_name ,segment_column_id ,internal_column_id ,hidden_column ,virtual_column ,user_generated FROM dba_tab_colsWHERE owner = 'OE' AND table_name = 'CURRCONV_PARTED'ORDER BY internal_column_id;TTITLE OFF
Viewing IDA Metadata
Metadata for Table OE.CURRENCY_CONVERSIONS (from DBA_TAB_COLS)
Seg Int Col Col UserColumn Name # # Hidden? Virtual? Gen'd?------------------------------ ----- ----- -------- -------- --------CURRENCY_FROM 1 1 NO NO YESCURRENCY_TO 2 2 NO NO YESEFFECT_DTM 3 3 NO NO YESCONVERSION_FCTR 4 4 NO NO YESSYS_NC00005$ 5 5 YES NO NOORA_ARCHIVE_STATE 6 6 YES NO NO
Note the addition of two new hidden
columns exclusively
for IDA
#C
14
LV
Populating IDA Test DataDECLARE ctr NUMBER := 0; curfr CHAR(3); curto CHAR(3); effdt DATE;BEGIN FOR ctr IN 1..36000 LOOP effdt := (TO_DATE('12/31/2015','mm/dd/yyyy') - (ctr/10)); INSERT INTO oe.currconv_parted VALUES( DECODE(MOD(ctr, 6), 0,'USD', 1,'JPY', 2,'EUR', 3,'AUD', 4,'CHF', 5, 'GBP') ,DECODE(MOD(ctr,36), 0,'JPY', 1,'EUR', 2,'AUD', 3,'CHF', 4,'GBP', 5,'CDN', 6,'EUR', 7,'AUD', 8,'CHF', 9,'GBP', 10,'CDN', 11,'USD', 12,'AUD', 13,'CHF', 14,'GBP', 15,'CDN', 16,'USD', 17,'JPY', 18,'CHF', 19,'GBP', 20,'CDN', 21,'USD', 22,'JPY', 23,'EUR', 24,'GBP', 25,'CDN', 26,'USD', 27,'JPY', 28,'EUR', 29,'AUD', 30,'CDN', 31,'USD', 32,'JPY', 33,'EUR', 34,'AUD', 35,'CHF') ,effdt ,ROUND(DBMS_RANDOM.VALUE(1,5),6) ); IF MOD(ctr, 5000) = 0 THEN COMMIT; END IF; END LOOP; COMMIT;EXCEPTION WHEN OTHERS THEN DBMS_OUTPUT.PUT_LINE('Fatal unexpected error has occurred: ' || SQLERRM );END;/
Generates 36,000 rows of historical test data for conversions between six key
currencies
SQL> UPDATE oe.currency_conversions SET ora_archive_state = 10 WHERE (currency_from = 'GBP') OR (currency_to = 'GBP'); COMMIT;
Activates IDA for any rows that
convert currencies from or to British
Pounds
#C
14
LV
Activating IDA “Cloaking”SQL> ALTER SESSION SET row archival visibility = ACTIVE;
COL currency_from FORMAT A08 HEADING "From|Currency"COL currency_to FORMAT A08 HEADING "To|Currency"COL effect_cdtm FORMAT A19 HEADING "Effective As Of:"COL conversion_fctr FORMAT 9999.999999 HEADING "Currency|Conversion|Factor"COL min_dtm FORMAT A19 HEADING "Minimum|Effective|Date"COL max_dtm FORMAT A19 HEADING "Maximum|Effective|Date"COL min_cvf FORMAT 9999.999999 HEADING "Minimum|Currency|Conversion|Factor"COL max_cvf FORMAT 9999.999999 HEADING "Maximum|Currency|Conversion|Factor"
TTITLE "Active Currency Conversion Factors|(from OE.CURRENCY_CONVERSIONS)"SELECT currency_from ,currency_to ,TO_CHAR(MIN(effect_dtm), 'yyyy-mm-dd hh24:mi') min_dtm ,TO_CHAR(MAX(effect_dtm), 'yyyy-mm-dd hh24:mi') max_dtm ,MIN(conversion_fctr) min_cvf ,MAX(conversion_fctr) max_cvf FROM oe.currconv_parted GROUP BY currency_from, currency_to ORDER BY currency_from, currency_to;TTITLE OFF
s
Active Currency Conversion Factors (from OE.CURRENCY_CONVERSIONS)
25 rows selected. No results for British Pound appear …
SQL> SELECT currency_from, currency_to, TO_CHAR(effect_dtm, 'yyyy-mm-dd hh24:mi'), conversion_fctr FROM oe.currency_conversions WHERE (currency_from = 'GBP') OR (currency_to = 'GBP') ORDER BY 1,2,3;
no rows selected… and all British Pound entries are “cloaked”!
#C
14
LV
Deactivating IDA “Cloaking”SQL> ALTER SESSION SET row archival visibility = ALL;
Active Currency Conversion Factors (from OE.CURRENCY_CONVERSIONS)
■TV allows specification of time periods when data stored within a table will be actually considered “valid”
■Implemented through new PERIOD FOR table attribute
■Specified TV dimension can then be populated as desired with both time-valid and non-time-valid data
■Non-time-valid data can be compressed until it’s no longer needed
#C
14
LV
Implementing TV: Examples
DROP TABLE oe.product_pricing PURGE;CREATE TABLE oe.product_pricing ( product_id NUMBER(6) NOT NULL ,unit_price NUMBER(10,6) DEFAULT 0 ,beg_dtm TIMESTAMP NOT NULL ,end_dtm TIMESTAMP NOT NULL ,PERIOD FOR effective (beg_dtm, end_dtm)) TABLESPACE example;
… or, if columns are specified
during the table’s creation, they’ll be used as the new dimension
columns
DROP TABLE oe.product_pricing PURGE;CREATE TABLE oe.product_pricing ( product_id NUMBER(6) NOT NULL ,unit_price NUMBER(10,6) DEFAULT 0 ,beg_dtm TIMESTAMP NOT NULL ,end_dtm TIMESTAMP NOT NULL ,PERIOD FOR effective) TABLESPACE example;
The new dimension can be specified
without any predefined columns,
and they will be created automatically
…
#C
14
LV
TV: Populating Sample DataDECLARE ctr NUMBER := 0; begdt DATE := TO_DATE('01/01/2000','mm/dd/yyyy'); enddt DATE;BEGIN FOR ctr IN 1..180 LOOP enddt := ADD_MONTHS(begdt, 1); FOR pidx IN 1..10 LOOP INSERT INTO oe.pricing_parted VALUES( DECODE(MOD(pidx,10) , 0,1726, 1,1729, 2,1733, 3,1734, 4,1737 , 5,1801, 6,2056, 7,2058, 8,3399, 9,3400, 3515) ,ROUND(DBMS_RANDOM.VALUE(1,100),4) ,begdt ,enddt ); END LOOP; begdt := enddt; END LOOP; COMMIT;EXCEPTION WHEN OTHERS THEN DBMS_OUTPUT.PUT_LINE('Fatal unexpected error has occurred: ' || SQLERRM );END;/
Adds 1,800 rows of randomized data for Temporal Validity testing
#C
14
LVLeveraging TV Session-Level Filtering
CONNECT / AS SYSDBAGRANT EXECUTE ON DBMS_FLASHBACK_ARCHIVE TO oe;
GRANTs access to users to request a
valid time window
-- Summary of all product prices:SELECT product_id ,TO_CHAR(MIN(beg_dtm), 'yyyy-mm-dd.hh24:mi:ss') start_dtm ,TO_CHAR(MAX(end_dtm), 'yyyy-mm-dd.hh24:mi:ss') stop_dtm ,MIN(unit_price) min_price ,MAX(unit_price) max_price FROM oe.pricing_parted GROUP BY product_id ORDER BY product_id;
-- Detail for a single product:SELECT product_id ,unit_price ,TO_CHAR(beg_dtm,'YYYY-MM-DD') beg_dte ,TO_CHAR(end_dtm,'YYYY-MM-DD') end_dte FROM oe.pricing_parted WHERE product_id = 1726 ORDER BY product_id, beg_dtm;
Summary of Unit Pricing Statistics Within Product ID (from OE.PRICING_PARTED) Starting EndingProduct Time Time Minimum Maximum # Entries Period Period Unit Price Unit Price------- ---------- ------------------- ------------------- ----------- ----------- 1726 180 2000-01-01.00:00:00 2015-01-01.00:00:00 1.18 129.08 1729 180 2000-01-01.00:00:00 2015-01-01.00:00:00 1.08 99.29. . . 3399 180 2000-01-01.00:00:00 2015-01-01.00:00:00 1.79 98.31 3400 180 2000-01-01.00:00:00 2015-01-01.00:00:00 1.68 99.99
1800 rows selected.
Detailed Product Pricing for 1726 (from OE.PRICING_PARTED) Starting EndingProduct Unit Time Time # Price Period Period------- ----------- ------------------- ------------------- 1726 85.26 2000-01-01.00:00:00 2000-02-01.00:00:00 1726 40.55 2000-02-01.00:00:00 2000-03-01.00:00:00. . . 1726 34.48 2014-10-01.00:00:00 2014-11-01.00:00:00 1726 34.24 2014-11-01.00:00:00 2014-12-01.00:00:00
Detailed Product Pricing for 1726 (from OE.PRICING_PARTED)
Starting EndingProduct Unit Time Time # Price Period Period------- ----------- ------------------- ------------------- 1726 53.32 2009-01-01.00:00:00 2009-02-01.00:00:00
Summary of Unit Pricing Statistics Within Product ID (from OE.PRICING_PARTED) Starting EndingProduct Time Time Minimum Maximum # Entries Period Period Unit Price Unit Price------- ---------- ------------------- ------------------- ----------- ----------- 1726 1 2014-03-01.00:00:00 2014-04-01.00:00:00 77.32 77.32 1729 1 2014-03-01.00:00:00 2014-04-01.00:00:00 42.89 42.89 1733 1 2014-03-01.00:00:00 2014-04-01.00:00:00 88.22 88.22 1734 1 2014-03-01.00:00:00 2014-04-01.00:00:00 51.49 51.49 1737 1 2014-03-01.00:00:00 2014-04-01.00:00:00 2.73 2.73 1801 1 2014-03-01.00:00:00 2014-04-01.00:00:00 1.88 1.88 2056 1 2014-03-01.00:00:00 2014-04-01.00:00:00 99.57 99.57 2058 1 2014-03-01.00:00:00 2014-04-01.00:00:00 78.32 78.32 3399 1 2014-03-01.00:00:00 2014-04-01.00:00:00 67.21 67.21 3400 1 2014-03-01.00:00:00 2014-04-01.00:00:00 3.16 3.16
Detailed Product Pricing for 1726 (from OE.PRICING_PARTED) Starting EndingProduct Unit Time Time # Price Period Period------- ----------- ------------------- ------------------- 1726 77.32 2014-03-01.00:00:00 2014-04-01.00:00:00
#C
14
LV
Temporal History (TH)
■Original name: Oracle Total Recall▪Later releases: Flashback Data Archive
■Intrinsic to Oracle 12cR1■Permits bi-temporal queries:
▪Flashback Query uses AS OF clause▪Flashback Versions Query uses VERSIONS
BETWEEN clause
#C
14
LV
Leveraging Bi-Temporal Queries
108.85 108.00 94.21 119.11 98.90 51.59 15.07
Changed on 04-05-2013 107.25 98.90
Changed on 04-18-2013 95.23 119.11
Changed on 04-30-2013 98.90
SCN
01-13
02-13
03-13
04-13
05-13
06-13
07-13
08-13
Time
EXEC DBMS_FLASHBACK_ARCHIVE(‘ASOF’,’2013-04-01’);SELECT product_id ,unit_price ,TO_CHAR(beg_dtm,'YYYY-MM-DD') beg_dte ,TO_CHAR(end_dtm,'YYYY-MM-DD') end_dte FROM oe.pricing_parted AS OF PERIOD FOR effective TO_DATE('2013-04-15','yyyy-mm-dd') WHERE product_id = 1726 ORDER BY product_id, beg_dtm;
Flashback Query
EXEC DBMS_FLASHBACK_ARCHIVE(‘ASOF’,’2013-04-01’);SELECT product_id ,unit_price ,TO_CHAR(beg_dtm,'YYYY-MM-DD') beg_dte ,TO_CHAR(end_dtm,'YYYY-MM-DD') end_dte FROM oe.pricing_parted VERSIONS PERIOD FOR effective BETWEEN TO_DATE('2013-04-08','yyyy-mm-dd') AND TO_DATE('2013-04-30','yyyy-mm-dd') WHERE product_id = 1726 ORDER BY product_id, beg_dtm;
Flashback Versions Query
#C
14
LV
Leveraging ILM ADO Features
#C
14
LV
Automatic Data Optimization (ADO)
■Moves or compresses data based on observed usage patterns
■Leverages heat maps to determine how often data has been accessed
■Tracks exactly how data has been utilized ▪DML versus query▪Random access versus table scan
■Usage patterns can be tracked at tablespace, segment, and even row level
#C
14
LV
Covering the Space-Time Continuum
An ADO policy can be based on either space or time■Space-based:
▪ Moves segment between tablespaces in different storage tiers
▪ Movement is based on “fullness” of initial tablespace
■Time-based:▪ Compresses data more tightly within same object▪ Data compressible at three levels:
— ROW STORAGE (row level within a segment)— SEGMENT (one database object at segment level)— GROUP (multiple database objects)
#C
14
LV
IDA: Implementing ILM Storage TieringCREATE table oe.currconv_parted ( currency_from VARCHAR2(3) NOT NULL ,currency_to VARCHAR2(3) NOT NULL ,effect_dtm TIMESTAMP NOT NULL ,conversion_fctr NUMBER(10,6) DEFAULT 0) PARTITION BY LIST(ora_archive_state) ( PARTITION currconv_history VALUES (10,20,30,40,50) TABLESPACE ado_cold_data ILM ADD POLICY COMPRESS FOR QUERY HIGH SEGMENT AFTER 180 DAYS OF NO MODIFICATION ,PARTITION currconv_current VALUES (0) TABLESPACE ado_hot_data ) ROW ARCHIVAL NOLOGGING PARALLEL 4;
After 180 days of no further DML activity, the segment of the partition containing all inactive rows
will be compressed
using HCC QUERY HIGH compression
#C
14
LV
TV: Implementing ILM Storage Tiering
CREATE TABLE oe.pricing_parted ( product_id NUMBER(6) NOT NULL ,unit_price NUMBER(10,6) DEFAULT 0 ,beg_dtm TIMESTAMP NOT NULL ,end_dtm TIMESTAMP NOT NULL ,PERIOD FOR EFFECTIVE (beg_dtm, end_dtm)) PARTITION BY RANGE(end_dtm) ( PARTITION opp_history VALUES LESS THAN (TO_DATE('2014-01-01','yyyy-mm-dd')) TABLESPACE ado_cold_data ILM ADD POLICY COMPRESS FOR QUERY HIGH SEGMENT AFTER 180 DAYS OF NO ACCESS ,PARTITION opp_current VALUES LESS THAN (MAXVALUE) TABLESPACE ado_warm_data ) NOLOGGING PARALLEL 4;
#C
14
LVOver To You …
#C
14
LV
Please feel free to evaluate this session:Use the COLLABORATE14 Application!
12c’s Information Lifecycle Management (ILM) Features for Improved Database Performance
If you have any questions or comments, feel free to:E-mail me at [email protected] my blog (Generally, It Depends):http://jimczuprynski.wordpress.comConnect with me on LinkedIn (Jim Czuprynski)Follow me on Twitter (@jczuprynski)