- 1. Information Means The World. Information Means The
World.
2. Storage: A New Paradigm for Databases Ari Kaplan President,
IOUG & Datalink Database Practice [email_address] 312-399-0079
3. Oracle: 30 years! IPOs: Sun: March 4, 1986 Oracle: March 12,
1986 Microsoft: March 13, 1986 4. 30 years of IT paradigms: -
Mainframe - Client-Server - Internet - Mobile - Web 2.0 Storage
paradigms: - Direct-attached disk - Tape backups - NAS -
Disk-to-disk - Virtualization - Replication 5.
-
-
- Oracle-based Methods:Hot backup, RMAN, Data Guard, Export /
Import, Data Pump, Flashback
-
-
- Snapshots :Enterprise Backup and Recovery
-
-
- Snapshots :Development and testing for cloning
-
-
- Enterprise Backup and Recovery :
-
-
-
- Disk-to-disk, enhanced data recovery, and VTL
-
-
- Disaster Recovery :storage-based replication
-
-
- Cost savings :tiered architecture / ILM
-
-
- Performance and high-availability :RAID-DP and aggregates
-
-
- Storage security / encryption
-
-
- Pros and Cons :Oracle and non-Oracle enterprise solutions
Agenda 6. A Silo Approach?
- Ive always focused on the Oracle side due to implementing RAC.
Every company Ive worked with has had their storage team take care
of that side of the house.
-
- One of Americas top Oracle RAC specialists
7. Databases Growing at Staggering Rate
- The WinterCorp survey found that the worlds largest databases
have posted an annual compounded growth rate of approximately 75%
since 1995.
Size of the Largest Data Warehouse in the WinterCorp TopTen
Survey 8. Storage Solutions for DBA Challenges
- Scalability:it can take a long time to physically copy 2 TB of
data to tape or disk
- Cost:it costs a lot to purchase 200 TB of storage so you can
perform physical image backups of 200 TB databases. It is costly to
even purchase hardware to test backups.
- Performance:keeping large databases in hot backup mode
negatively affects the performance of high-transaction systems
(inserts, updates, deletes)
- Complexityof systems: multiple databases, interlinked systems,
different database versions, RMAN/non-RMAN, RAC, ASM, etc.
- Manageability:it is often difficult to setup, manage, and test
backups
- Manageability:human errors, lost data, inconsistent data,
physical failures, corruption can require restores. It can be a
manual and scary process to recover to a consistent
point-in-time
- Performance:how in the world do you recover a 2 TB database in
15 minutes?
- Disaster Recovery / Replication
- How do you architect your database and surrounding environment
for DR? With no data loss? With a 15-minute failover
timeframe?
- Cost:DBAs tend to put storage on single class of storage
without archiving or tiering considerations
- Performance:system response time is 5 seconds now. What happens
when my data triples in size?
- Manageability:it is often a political process to get additional
storage from non-DBA groups
- Cost:it costs a lot to purchase 20 TB of storage to get several
image copies of production to test and development!
- Scalability:it is too much to provide 5, 10, 15 or more copies
to development and testing teams
- Manageability:it can take 25% or more of a DBAs time to manage
the cloning process
9. RPO and RTO Recovery Point Database-only is typically days or
hours of lost data in disaster, or too costly. Combined with
storage can be faster, heterogeneous, and with less cost Recovery
Time Database-only can be days or hours. Combined with storage can
be minutes. Backup Window Database-only is typically days for
terabyte-sizes. Combined with storage can be seconds Tape Backup
Hot backup Tape Backup Hot backup RMAN Data Guard Copy-on-Write
Snapshots WAFL Snapshots Async replication RMAN Synchronous
replication,Data Guard 10.
-
-
- Snapshots: Enterprise Backup and Recovery
11. Oracle Tablespace Architecture datafile 1 datafile 2
datafile 3 datafile 4 datafile 5 Database Blocks Extent 28K Extent
84K Segment 112K (Table, Index, etc.) TABLESPACE 12. Oracle
Database Architecture Image from Oracle Corp 13. Oracle Backup
Methods: Hot Backups
- Online backup of the database
- Can be used with other backup methods
- Can be prohibitively lengthy
- Only backs up some of the Oracle files not customized code or
non-Oracle systems such as Exchange, applications, and SQL
- Need an automated script that looks at Oracles configuration on
the fly
14. Oracle Backup Methods: RMAN
- Block-level backups - datafile compression
- Many companies see a 10x + performance depending on the amount
of updates that were made between backups
- Use of arecovery catalogfor multiple databases
15. Oracle Backup Methods: RMAN
- Significantly reduces RTO
- Significantly less storage space for incremental backups (only
changed blocks)
- Parallel streams/channels
- Works well with snapshots, SnapMirror, SMO, Decru, future
de-duplication
- RMAN only backs up the database, not code or non-Oracle
systems
16. Oracle Backup Methods: Data Guard
- Replicates Oracle databases from one data center to
another
- Performs backups from the standby database instead of the
production database
- Both physical and logical versions
17. Oracle Backup Methods: Data Guard
- Not supported by earlier versions of Oracle
- Only supports Oracle databases, not source code or non-Oracle
systems
- Standby database must be online
18. Oracle Backup Methods: Export/Import
- Logical backups of tables (rather than copying the physical
blocks of data, it copies the series of commands used to recreate
the tables)
- Can recover on a table-by-table basis instead of the entire
database
- Good for complementing other backup methods
- Can take a LONG time versus snapshot methods
- All data since the export was made is lost
- A database must be running to perform either an export or
import
- Export files shouldnt be edited and can only be used by
Oracle
19. Oracle Backup Methods: Export/Import
- Using the SELECT clause (compared to FlexClone)
- Logical copy of tables or the entire database
- Moving tables / data among schemas
- Moving tables / data among databases
- Recreating CREATE statements
-
-
- imp full=y show=y log=cr_db.sql (its ugly)
- Migrate among Oracle versions
- Note: some OSs have a 2G file-size limit so DBAs use the split
and/or compress / tar commands
20. Oracle Backup Methods: Data Pump
- Export/Imports replacement with significant improvements
- 15-45 times faster than Export/Import
- Suspend and restart data transfers
21. Oracle Backup Methods: Data Pump
- Like the Export/Import utilities
-
- All data since the Data Pump started is lost
-
- Still can take a long time to back up and recover versus
methods such as snapshots
-
- A database must be running to perform either an export or
import
- Data Pump does NOT yet work with XML schemas and XML
schema-based tables
- When you are importing data into an existing table using either
APPEND or TRUNCATE, if any row violates an active constraint, the
load is discontinued and no data is loaded
- Situations where external tables is used and Direct Path Load
is not
-
- there is an active trigger
-
- a referential integrity constraint exists
-
- fine-grained access control is enabled in insert mode
-
- the table has encrypted columns, table is in a cluster
-
- a global index on multi-partition tables exists during a
single-partition load
-
- a domain index exists for an LOB column
-
- a table contains BFILE columns or columns of opaque types
-
- a table contains VARRAY columns with an embedded opaque
type
-
- supplemental logging is enabled and the table has an LOB
column
22.
- Oracles flashback area allows for recovering a table (or
database) to any point-in-time in the past by storing images of
data online
- Recover to point-in-time from Flash Recovery Area
- FLASHBACK TABLE LAOUG_audience,
- free_gift_list TO TIMESTAMP
Oracle Backup Methods: Oracle Flashback 23.
- DROP command puts object into a recycle bin for quick
recovery
Oracle Backup Methods: Oracle Flashback 24. Oracle Backup
Methods: Oracle Flashback
- This method provides online backup and recovery, eliminating
the need to recover from tape and saving valuable recovery time and
management effort
- Extremely fast recovery of tables with simple commands
- Requires a significant amount of flashback area online, taking
up lots of storage
- Cant recover from media failure
- Cant undo operations such asshrink datafile
- Can only flashback to the oldest SCN in the FRA
- Only works with newer releases of Oracle
25. Storage Backup Methods
- Non-Oracle Storage Backup Solutions
- Non-Oracle solutions are sometimes undiscovered in the world of
DBAs
- Can solve issues that simply cannot be done with Oracle-only
technology
- May already be deployed in a company to support
Oracle/non-Oracle applications but the database staff is unaware of
its HUGE benefits to database environments
26. Storage Backup Methods: Triple-Mirroring
- Copies the data in real time to three sets of redundant
disks
- Its possible to slice the mirror instantaneously and back up
from the slices, thereby eliminating performance hits of being in
hot backup mode for extended periods of time
- Expensive: 50% more storage costs to keep a set of
production-sized disks
- Companies still may not be able to meet backup windows,
especially if backing up from the mirror takes more than 24
hours
27. Storage Backup Methods: Array-Based Replication
- Replication between two storage arrays that sends storage layer
blocks to a standby site whenever there is a storage change at the
primary site
- Fills the gaps of Data Guard by replicating non-Oracle systems
and source code as well as tables in NOLOGGING mode
- Works best in conjunction with Data Guard by jointly reducing
replication traffic and reducing or eliminating single points of
failure
- Significantly improves RTO and RPO
- Source and target may need to be the same vendor, unless using
heterogeneous replication such as Topio
- There is a cost associated with purchasing and maintaining a
third-party array-based replication solution. An ROI analysis will
demonstrate that this cost can be justified. The point at which it
is recouped will depend on the value of the data
28.
- All of these should be backed up!
- Datafiles data, index, temporary, rollback, system
- Online and archived redo logs
- Oracle executables and patches
- Export, Data Pump dump, RMAN files
- Parameter files (init.ora, sqlnet.ora, listener.ora,
tnsnames.ora)
- Alert logs, bdumps, cdumps, udumps
- LOB or BFILE or library storage structures
Oracle Physical Files 29. Simpler Backup and Recovery Active
File System (CURRENT) All Oracle Datafiles 7 AM 8 AM 9 AM 10 AM
11AM Redo Logs Archive Logs Ctl Files 30. Storage Backup Methods:
Snapshots
- Instant backup and recovery of large data sets using a
sophisticated, scalable, and fail-safe pointer system of storage
blocks. Snapshots represent a frozen view of data taken at a
specific point-in-time. Data and entire environments can be
restored to a known stable point prior to the event that caused the
disruption or corruption.
C Snapshot.0 A B C Active File System (Changed blocks between
snapshots are tracked) Snapshot.0 file system version of ORACLE.DBF
is still composed of blocks A, B & C Snapshot.1 file system
version of ORACLE.DBF is composed of blocks A, B & C Snapshot.1
File: ORACLE.DBF File: ORACLE.DBF File: ORACLE.DBF 31. Technology
Integration Snapshot for Backup/DR validated by Oracle
- TPC-C Published with 5 active snapshots
Snapshot taken 32. Real World Snapshot Performance "In our test
configurations, we found that conducting a single snapshot copy on
the FAS3070 over the course of the test had no sustained impact on
the overall performance. On the CX3-80, creating a single snapshot
caused the overall performance level to drop to approximately 50
percent of the baseline performance. Additionally, the results show
that the post snapshot performance did not recover to levels
observed before the snapshot was taken. VeriTest Report, NetApp
FAS3070 vs. EMC CX3-80:Nov. 2006 33. Storage Backup Methods:
SnapManager for Oracle
- GUI that uses Snapshots for database backups, recoveries, and
cloning. Easy to install and integrates with Oracle RMAN, Oracle
RAC, Oracle ASM. Oracle 9i and 10g
- Greatly improves DBA efficiency in setting up test and dev
environments
- Automates DBA tasks and reduces risks for:
-
- Error-free backups and recoveries
- Initiate and manage backups
- Minimize the risk of data loss and corruption with increased
backup frequency
- Near instantaneous restores of data when its most critically
needed
- There is a cost associated with purchasing non-Oracle
solutions. As with the other storage-based methods, an ROI analysis
will demonstrate that this cost can be quickly justified. The point
at which it is recouped will depend on the value of the data and
the value of the speed of backup or recovery, improvements in
manageability, etc.
34.
-
-
- Snapshots: Development and testing for Cloning
35. Using Snapshot Cloning to Accelerate Application Development
and Upgrade Cycle
- Quickly reconfigure multiple test, development, QA, DW,
auditing, staging environments
- Rapid restore from online Snapshot copies
- Allows multiple test environments
- Helps deliver new Oracle applications quickly
- Use Oracle cloning procedures to clone application
environments
Test and Development Lab Oracle Server Test Database Clustered
Filers Database TestingCycle Only Takes Seconds Testing Baseline
Run Test Evaluate Results Restore Base Environment 36. Before:
Large E-Business Suite Production
- Copies consume lots of disk
-
- < 10% data differences for each instance
Mirrored Copy Test 1 Dev N Test N Test 2 Dev 1 Dev 2 37. After,
with Snapshot Clones Production Test 1 Test 2 QA Develop 1 Develop
2 Mirrored Copy
38. Oracle University and Cloning
- Refresh about 100 classes with 11 databases each weekend
- Entire operation now complete in 45 minutes!
39. Storage Backup Methods: SnapShots for Cloning (SMO and
FlexClone) Demo (05:04) Known Performance and Cost Unpredictable
Performance and Cost Scalability by Metric: To avoid surprise,
require proof of scalability at your requirement levels Test and
dev with full size database Test and dev with full size database)
Test and dev with subset of database 40. Storage Backup Methods:
SnapManager for Oracle
- GUI that uses snapshots for database backups, recoveries, and
cloning
- Easy to install and integrates with Oracle RMAN, Oracle RAC,
Oracle ASM, Oracle 9i and 10g
- Greatly improves DBA efficiency in setting up test and dev
environments
- Automates DBA tasks and enables error-free backups and
recoveries, restores and clones
- Initiates and manages backups
- Minimizes the risk of data loss and corruption with increased
backup frequency
- Provides near instantaneous restores
- Cost associated with purchasing non-Oracle products
-
- ROI analysis will demonstrate that this cost can be
justified
41.
-
-
- Disk-to-disk, enhanced data recovery and VTL
42. Disk-to-Disk: Enhanced Data Recovery is the Solution
- Pressure to reduce recovery time (RTO)
- Pressure to minimize data loss (RPO)
- Increasing data sizes / shrinking backup window
- Tape backup & restore is unreliable
- Media costs & media management complexity getting out of
control
43. Disk-to-Disk: Enhanced Data Recovery is the Solution
- A backup and restore architecture that adds a disk based
storage array to a traditional tape only solution.
- Enables concept of Backup to disk, Archive to tape
- Enabled by falling disk prices (ATA & SATA)
- Leverages random access capability of disk
- Augments tape capabilities
44. VTL: Efficient Backup and Restore
- VTL emulates industry standard tape libraries
- Time to restore is improved with online and near-line
backups
- Consolidated backups: integrates as appliance!
- Replace tape backups with VTL backups
- Rapid failover supported when a server fails you can remount
the database volumes to another server
Tape Library for backup WindowsServers Backup Server NearStore
VTL Storage UNIX Servers Decru DataFort 45.
-
-
- Disaster Recovery: Storage-based Replication
46. Replication Based Backup Solution ROI/TCO
- Replication between two storage arrays that sends storage layer
blocks to a standby site whenever there is a storage change at the
primary site
- Reduces the number of copies of data backed up
- Reduces IP network traffic for backup data
- More frequent backups with more copies kept online
- Reduces tape media and automation resources
- Fast file & full restores
- Centralization of remote backup
- Replication to off-site location lowers tape media management
costs for off-site tape rotation
47. Oracle Backup Methods: Data Guard
- Replicates Oracle databases from one data center to
another
- Performs backups from the standby database instead of the
production database
- Both physical and logical versions
48. Oracle Backup Methods: Data Guard
- Not supported by earlier versions of Oracle
- Only supports Oracle databases, not source code or non-Oracle
systems
- Standby database must be online
49. Storage-based Replication for Oracle
- Oracle DB volumes on primary and standby site are
SnapMirrored
- Remote sites Oracle DB is kept in recovery mode
- When disaster strikes primary site, its a matter of breaking
the SnapMirror and bringing up the remote standby sites Oracle DB
in query/update mode
GigE / FCP /vol/Oracle Data, Log & Cntrl files /vol/Oracle
Mirrored Data, Log andCntrl files Asynchronous replicationover
TCP/IP LAN or WAN. No distance limitation. Primary Site remote disk
mirroring solutions NAS/SAN StorageAppliance Changed blocks are
shipped to the target mirrored volume Standby Site NAS/SAN
StorageAppliance GigE / FCP NYC Chicago 50. Storage Backup Methods:
Array-Based Replication
- Fills the gaps of Data Guard by replicating non-Oracle systems
and source code as well as tables inNOLOGGINGmode
- Works best in conjunction with Data Guard by jointly reducing
replication traffic and reducing or eliminating single points of
failure
- Significantly improves RTO and RPO
- Source and target may need to be the same vendor, unless using
heterogeneous replication such as Topio
- There is a cost associated with purchasing and maintaining a
third-party array-based replication solution. An ROI analysis will
demonstrate that this cost can be justified. The point at which it
is recouped will depend on the value of the data
51.
-
-
- Cost savings: tiered architecture / ILM
52. Oracle and Virtualization: Enterprise Grid
- Scalable everything: performance, availability, capacity,
etc.
- Single view; single point of control
Pooled Storage Network with TransparentReconfiguration:
Compliance Database Backup Data Secure Data CPUs OS andLoadSharing
Software Single Virtual Machine High Access Data on Primary Storage
Single Virtual Storage Pool Global Name Space andUnified Management
53. Tiered Architecture / ILM
- Oracle options for tiering data
- Tablespaces on specific tiers of disks
- Table and index partitioning tiers of disk relate to specific
partitions
- Triggers, procedures, packages for migrating data
- ILM solutions from Oracle or third-parties
-
-
- Create Storage Tiers for the Data Classes
-
-
- Create Data Access and Migration Policies
-
-
- Define and Enforce Compliance Policies
-
- Third-party solution integration with Oracle
54. Cost savings example: 94% reduction in storage costs!
- Oracle options for tiering data
- Tablespaces on specific tiers of disks
- Table and index partitioning tiers of disk relate to specific
partitions
- Triggers, procedures, packages for migrating data
- ILM solutions from Oracle or third-parties
-
-
- Create Storage Tiers for the Data Classes
-
-
- Create Data Access and Migration Policies
-
-
- Define and Enforce Compliance Policies
55.
-
-
- Performance and high-availability: RAID-DP
56. RAID-DP
- RAID-DP is an advanced, cost-effective disk failure/error
protection solution protecting information in the event of a double
disk or media failure within a single RAID group
- RAID-DP is based on RAID4 adding a diagonal parity calculation
to enhance overall performance versus competitive double parity
adaptations based on RAID6
3 1 2 1 1 1 3 1 2 2 1 3 3 1 2 2 9 5 8 7 7 12 12 11 { D D D D P
DP 57. Why is RAID-DP Needed?
- Traditional single-parity-drive RAID group no longer provides
enough protection
-
- Reasonably-sized RAID groups (e.g. 8 drives) are exposed to
data loss during reconstruction
- RAID-DPs double disk-failure protection does what RAID5 and
RAID1/0 cannot:
-
- Reduces RISK:limits exposure to same RAID group second disk
failure or non-recoverable media error
- RAID 1 is too costly for widespread use
-
- Mirroring doubles the cost of storage
-
- Not affordable for all data
- RAID-DP exceeds RAID1/0 protection levels without the
associated doubling of capacity and cost
- RAID-DP has a comparable operational cost to RAID 4
- Optimized for Performance
- Reduces RAID group rebuild time
58. RAID-DP vs other RAIDs
- >10,000 times more secure than single-parity RAID
- More reliable than mirroring for double-disk failure
- 13% Parity Overhead vs50% Overhead w/ Mirror (*)
- 75% more usable capacity than competitive offerings (*)
- * Comparing 14d + 2p vs. 8+8 mirror
P P DP RAID Protects against single disk failure RAID DPProtects
against anytwo-disk failure 59. The Cost of Data Availability &
Protection Compare RAID-DP to RAID10 Count the drives needed for
2TB useable storage using 144GB disk drives RAID-DP RAID-DP
protects as well as RAID10 with less storage overhead RAID 10 30 17
60. A Simpler Database Architecture Redo Logs Archive Logs Ctl
Files All Oracle Datafiles 61. Aggregates: a Simpler Database
Design RAID-DP Multi-Disk Aggregate Database 62. Aggregates:
Improved Performance Test OLTPDatabase - 1 Test OLTPDatabase - 2
Transaction log Archive log Flashback Data files All files 24 Disk
Aggregate 32 Disk Aggregate 8 Disk Aggregate 63. Aggregates:
Improved Performance Test OLTPDatabase - 1 Test OLTPDatabase - 2
Transaction log Archive log Flashback Data files All files 24 Disk
Aggregate 32 Disk Aggregate 8 Disk Aggregate 64.
65. Other Backup Methods: De-duplication
- De-duplication works with VTLs by detecting redundant data
streams during the backup process and sending pointers to that data
(versus actual streams of blocks of data) when duplicate streams
are detected
De-dupe device 66. Other Backup Methods: De-duplication
- Can dramatically reduce the backup stream sizetypically 10-20
timesand bandwidth required for backup
- 10-40% lower acquisition cost than tape
- When set up properly, de-duplication can work well with
multiple Oracle RMAN channels
67.
68. Encryption by Storage Vendors
- Unencrypted data opens up the company to several
vulnerabilitieswhether its a disgruntled employee or consultant
stealing the tape or the tape being lost in the warehouse or while
being transported off-site.
69.
-
- Store data encrypted within database tables and, thus, also
encrypted on tape
-
- Data may be unencrypted in indexes, temp tablespace, flashback,
audit, and elsewhere
-
- Encrypt Oracles RMAN backups before they are written to
tape
-
- By encrypting at the database level, an organization reduces
much of itsexposure. This method is also free (for one
direct-attached storage device)
-
- Typically, the DBA is solely responsible for retaining the key
and there areinherent weaknesses with this strategy
- 2)Storage vendor encryption
- Encryption at the storage level
- Multi-master key management that prevents single
points-of-failure for key loss
- No auto-destruct capabilities
- By using a hardware and software appliance, backup streams can
be encrypted with negligible impact on performance
Encryption by Oracle 70. Recovery Methodologies
- An untested or undocumented backup and recovery plan is no plan
at all
71. Summary, Q & A
72. Addendum Slides 73. Datalink Overview
- Independent information storage architect specializing in
solutions & services spanning:
- Focused on information storage solutions and services since
1987
- Headquartered in Minneapolis, MN, with operations throughout
the U.S.
- Publicly traded company (NASDAQ: DTLK)
74. Datalink Value Proposition
-
- Business needs drive recommendations
- Storage focus & expertise
-
- Insiders view of manufacturer R&D roadmaps
-
- Extensive experience with complex, multi-vendor, multi-platform
infrastructures
-
- Alignment of people, processes and technologies
- Track record of excellence