Application MAA Best Practices on Oracle Private Cloud Darryl Presley, MAA Team, CMTS Lyn Pratt, MAA Team, CMTS
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 1
Application MAA Best Practices on Oracle Private Cloud Darryl Presley, MAA Team, CMTS Lyn Pratt, MAA Team, CMTS
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 2
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decision. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Safe Harbor Statement
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 3
Application MAA Best Practices on Oracle Private Cloud Drivers for Consolidation
Maximum Availability Architecture
Oracle Engineered Systems
Applications Unlimited on the Private Cloud
Consolidation Design Principles
Case Studies
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 4
Drivers for Consolidation
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 5
Legacy deployments
Heterogeneous Per-project deployments Low utilization Hard to manage and govern Slow to deploy and change
High risk, high cost, inflexible
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 6
Key Goals • Reduce variety and complexity • Standardize at all levels • Simplify
How to do it • Fewest possible products, versions, vendors • Modular configurations • Pre-defined placement policies • Pre-defined management processes • Plan for expansion and exceptions
Drivers for Consolidation – Reduce Risk
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 7
Key Goals
• Reduce resources and environments • Maximize utilization
How to do it
• Provision as little as possible • Consolidate as much as possible • Pre-defined co-location and fullness policies • Isolate only as needed
Drivers for Consolidation – Reduce Costs
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 8
Key Goals • Rapid deployment and provisioning • Fewer manual processes • Online IT management
How to do it • Modular, pre-built components: Engineered Systems • Pay-as-your-grow platforms • Dynamic footprint (e.g., QoS Management) • Rolling upgrades • Self-service
Drivers for Consolidation – Increase Agility
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 9
Maximum Availability Architecture
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 10
RAC – Scalability – Server HA
ASM – Volume Management Online Redefinition,
Edition Based Redefinition, Data Guard, GoldenGate – Minimal downtime maintenance, upgrades, and migrations
Production Site
Maximum Availability Architecture (MAA) Low-Cost, Integrated, Fully Active, High ROI
Active Data Guard – Data Protection, DR – Query Offload
GoldenGate – Active-active – Heterogeneous
Oracle Secure Backup – Backup to tape / cloud
Active Replica
Flashback – Human error
correction
RMAN & Fast Recovery Area – On-disk backups
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 11
Online Redefinition, Edition Based Redefinition, Data Guard, GoldenGate – Minimal downtime maintenance, upgrades, and migrations
Production Site
RAC – Scalability – Server HA
Flashback – Human error
correction
Active Data Guard – Data Protection, DR – Query Offload
GoldenGate – Active-active – Heterogeneous
Active Replica
Maximum Availability Architecture (MAA) Low-Cost, Integrated, Fully Active, High ROI
Oracle Secure Backup – Backup to tape / cloud
ASM – Volume Management
RMAN & Fast Recovery Area – On-disk backups
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 12
Data Failure Human Error
Hardware Failure Site Disaster
Software Failure
UNPLANNED DOWNTIME Failures & Solutions
Backup & Recovery
Death Detection and restart Clusters & Load Balancing Server/Service Migration State Replication and Replica aware Stubs
SiteGuard Clusters & Load Balancing Server Migration Clusterware Integration
Maximum Availability Architecture
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 13
Deploy and Re-deploy Applications
Transformations, scalability and topology extensions
Patching
Configuration Changes
PLANNED DOWNTIME Operations & Solutions
Hot Deployment Side By Side Deployment
•Online configuration Changes • Changes and warnings •Batching changes / Deferred Activation
Rolling Patching
•Cluster wide JNDI •Dynamic Clusters
Maximum Availability Architecture
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 14
Oracle Engineered Systems
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 15
Engineered Systems I The Family Industry “Positive Disrupters”
• Expedited time to value • Easier to manage and upgrade • Lower cost of ownership
Exalytics Exadata Database Machine
Exalogic Elastic Cloud
Database Appliance
SPARC SuperCluster
Big Data Appliance
• Reduced change management risk • One-stop support • Extreme performance
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 16
Exadata Hardware Architecture X3’s
• Database Grid – 8x 2-socket, or 2x 8-socket Xeon database servers – Oracle Linux or Solaris 11 – Oracle Database 11g, ASM, RAC – 10 Gb and 1Gb Ethernet (to data center)
• Intelligent Storage Grid – 2-socket storage servers – Up to 504 terabytes raw disk per rack – 22.4 terabytes PCI Flash storage per rack – Exadata Storage Server Software
• InfiniBand Network – Internal unified connectivity ( 40 Gb/sec )
Complete Database Grid using standard servers for Compute and Storage
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 17
Exalogic Hardware Architecture
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 18
Applications Unlimited on the Private Cloud
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 19
Making It Simpler…
Engineered systems come with redundancy built in OneCommand lays down the groundwork with MAA best practices
– Cluster services – Voting disks, Oracle Cluster Repository – SCAN listener – ASM – SAME striping methodology
Standard MAA deployments are executed the same way on these platforms as on any other – nothing new to learn
Integrated HA
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 20
Exalogic Elastic Cloud – PIA Web Servers – Application Server – Process Scheduler – PeopleSoft Report
Repository File System
Exadata Database Machine
– RAC Database Servers – Exadata Storage
Shared InfiniBand Fabric
PeopleSoft on Exalogic and Exadata
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 21
Siebel on Exalogic and Exadata Exalogic Elastic Cloud
– Web Servers – Siebel Gateway Servers – Siebel Servers – Siebel File System
Exadata Database Machine
– RAC Database Servers – Exadata Storage
Shared InfiniBand Fabric
ClusteredGatewayServer
OracleRAC and
ASM
OracleDatabase
Load Balancer
Web Servers
Siebel Servers
DB Servers
Storage
Siebel Servers
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 22
Forms Server
OracleRAC and
ASM
OracleDatabase
Concurrent Manager
PCP
Load Balancer
Application Tier
DB Servers
Storage
Self-Service Apps
Exalogic Elastic Cloud – Self Service Apps Web
Servers – Forms Servers – Concurrent Manager
(PCP) Servers – Apps tier File System
Exadata Database Machine
– RAC Database Servers – Exadata Storage
Shared InfiniBand Fabric
E-Business Suite on Exalogic and Exadata
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 23
MAA on the Private Cloud
Primary Site Secondary Site
Exadata
ActiveData Guard
StandbyDatabase
DB Servers
Web Servers
Middle Tiers
EBS Middle Tiers
PeopleSoft Middle Tiers
Exalogic
EBS DB Instances
Exadata
Exalogic
Siebel Middle Tiers
EBS Middle Tiers
PeopleSoft Middle Tiers
Siebel Middle Tiers
PeopleSoft DB Instances
Siebel DB Instances
EBS DB Instances
PeopleSoft DB Instances
Siebel DB Instances
File SynchronizationzFS Snapshots
Oracle Data Guard
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 24
Consolidation Design Principles
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 25
Consolidation Design Principles
Match HA requirements
Match maintenance windows
Balance the calendar
Leave wiggle room
Consolidate rationally
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 26
Consolidation Design Principles
Cross organizational boundaries?
Same environment?
Potential version incompatibilities?
Common security
requirements? Dev/test with production?
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 27
Consolidation Design Principles
Dev/ Test
Classes of systems
Mission critical
Departmental
Sometimes consolidated
onto DR hardware
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 28
Consolidation Design Principles
Two workloads to be brought into a pool What should we look for, to merge
the processing calendars?
Balancing System Requirements
Util
izat
ion
Time
A
Time
Util
izat
ion B
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 29
Consolidation Design Principles
Existing Workload
Time
Util
izat
ion
Peak
Average The smaller this gap
the better
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 30
Consolidate Rationally – Poor Match
Time
Util
izat
ion Time
Util
izat
ion
Util
izat
ion
Time
New Workload A
B
Existing Workload Peak
Average
Time
Util
izat
ion
Resulting Workload Antagonistic!
Difference between Average & Peak
Increases
Peak
Average
- or -
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 31
Consolidate Rationally – Good Match
Time
Util
izat
ion
Util
izat
ion
Time
New Workload A
B
- or -
Existing Workload
Time U
tiliz
ati
on
Peak Average
Util
izat
ion
Time
Peak
Average
Resulting Workload
Complementary! Difference between
Average & Peak decreases
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 32
Application MAA Best Practices on Oracle Private Cloud
Case Studies – Consolidating Disaster Recovery (DR) with Dev/Test – Using Smart Scans with OLTP – Planned Maintenance Patterns
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 33
Consolidating DR with Dev/Test
MAA best practice: – Mirror your production install in your DR data center, dedicate it to DR, add
reporting and backups at the standby via Active Data Guard – Separate dev/test/QA/performance environments
A common compromise implementation: – Deploy disaster recovery standby databases on the same machines as
dev/test/QA environments – Caveat: it’s still quite useful to have a test machine
Consolidating DR with Dev, Test, QA
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 34
Consolidating DR with Dev/Test
Large manufacturing company Keep database CPU and memory same in production and DR/dev/test
site – Ensures capacity for running production on failover – Provides adequate capacity for dev/test with planning
High performance disk in production, high capacity at DR/dev/test site – High redundancy on all disk groups so can roll in cell upgrades everywhere
Consolidating DR with Dev, Test, QA – Case Study
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 35
Consolidating DR with Dev/Test
Provide required IOPS on failover: – Added storage expansion rack to DR – Created DATA and RECO disk groups for
DR/performance databases Defined these first Placed voting disk in DATA
– Created DATA_TST and RECO_TST disk groups for dev/test databases, using remaining space (up to DBFS).
Consolidating DR with Dev, Test, QA – Case Study
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 36
Using Smart Scans with OLTP
I want to: – Have smart scans and quick OLTP
queries play well together – Take backups that don’t affect user
performance – Make sure my standby MRP
(Managed Recovery Process) keeps up with production
– Have my cake and eat it, too …
Co-mingling OLTP, Smart Scans, Backups, MRP
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 37
Using Smart Scans with OLTP
Goal: Execute smart scans when have capacity, but give small IOs priority so user response time not affected Implement IORM “balanced” or “low latency” objective
– Limit large IOs to 90% (balanced) or 50% (low latency) of capacity – Give small IOs preference if disks are busy – Not necessary to get fancy with splitting out resource allocations
Must map RMAN activity to specific resource to have IORM manage its large IOs
Make Small IOs “King” in Production
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 38
Using Smart Scans with OLTP
Create IORM plan on all the cells, setting the objective to ‘balanced’ – Limits large IOs to 90% capacity, gives small IOs preference if disks busy
Manage backups – Disable the database scheduler windows used by the
DEFAULT_MAINTENANCE_PLAN
Prevents disabling RMAN_THROTTLE database resource plan – Create RMAN_THROTTLE database resource plan and set it as the default. – Create database services managed by clusterware for each database, to
ensure the backup work is executed on each DB node – Write RMAN script to allocate channels to above services
Make Small IOs “King” in Production – Tasks
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 39
Using Smart Scans with OLTP
In addition to the work in the prior slide, alter the IORMPLAN to ensure the standby databases have first priority for system resources, then the test/dev databases
Make Sure MRP Wins at the DR Site
Level 0 Level 1 Level 2 Level 3 Sys: 100%
Prod A, Prod B, role=primary, 100%
Prod A 75%, Prod B 25%, role=standby
“Other” 100%
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 40
Planned Maintenance Patterns
Study the documentation for your upgrade. Plan carefully, test. Stage the file system bits ahead of the maintenance window / do out-
of-place patching where possible Roll OS and Grid Infrastructure upgrades and patches across DB
nodes Roll database patches across RAC nodes, upgrades using transient
Logical Standby Roll IB upgrades across the switches – wait between the switches! Roll cell upgrades
– implement high redundancy if planning to do rolling cell upgrades
General Best Practices for High Availability
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 41
Planned Maintenance Patterns
Application constraints and service level agreements may lead to other approaches:
– It may be impractical to roll upgrades across nodes for some applications, leading to non-rolling upgrade of OS, grid, DB homes
– Other applications may require 100% uptime
Conflicts
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 42
Planned Maintenance Patterns
Switch customer-facing applications to the Data Guard standby Shut down primary frame, do platform upgrades in parallel to reduce
maintenance window for other applications – OS in parallel – Grid: first server alone, all but one server in parallel, last server alone – Database: out of place patching.
Transient logical for upgrades, for customer-facing applications. Switch customer-facing applications back to primary frame Cell and IB switch patching either in parallel during maintenance
window or rolling outside maintenance window
Alternatives – Case Study
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 43
Resources
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 44
Resources OTN HA Portal:
http://www.oracle.com/goto/availability
Maximum Availability Architecture (MAA): http://www.oracle.com/goto/maa
MAA Blogs: http://blogs.oracle.com/maa
Exadata on OTN: http://www.oracle.com/technetwork/database/exadata/index.html
Oracle HA Customer Success Stories on OTN: http://www.oracle.com/technetwork/database/features/ha-casestudies-098033.html
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 45
Resources Oracle E-Business Suite
– Business Continuity using Oracle 11g Physical Standby Database https://support.oracle.com/epmos/faces/ui/km/DocumentDisplay.jspx?id=1070033.1
– E-Business Suite on Exadata http://www.oracle.com/technetwork/database/features/availability/maa-ebs-exadata-197298.pdf
– Patches Required with Oracle Database 11gR2 on Exadata https://support.oracle.com/epmos/faces/ui/km/DocumentDisplay.jspx?id=1392527.1
Oracle PeopleSoft
– MAA Best Practices http://www.oracle.com/technetwork/database/features/availability/maa-peoplesoft-bestpractices-134154.pdf
– Reducing Downtime with a Local Data Guard Standby Database http://www.oracle.com/technetwork/database/features/availability/maa-peoplesoft-local-standby-128609.pdf
– PeopleSoft on Exadata Database Machine http://www.oracle.com/technetwork/database/features/availability/maa-wp-peoplesoft-on-exadata-321604.pdf
Oracle Siebel
– MAA Best Practices http://www.oracle.com/technetwork/database/features/availability/siebelmaa-131211.pdf
– Reducing Downtime with a Local Data Guard Standby Database http://www.oracle.com/technetwork/database/features/availability/maa-wp-siebel-localstandby-1-131425.pdf
– Siebel on Exadata Database Machine http://www.oracle.com/technetwork/database/features/availability/maa-wp-siebel-exadata-177506.pdf
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 46
Resources Exalogic
– Disaster Recovery for Oracle Exalogic Cloud http://www.oracle.com/technetwork/database/availability/maa-exalogic-dr-401789.pdf
– Backup and Recovery Best Practices http://www.oracle.com/technetwork/database/availability/maa-exalogic-dr-401789.pdf
MOS Notes to Track
– Database Machine and Exadata Storage Server 11g Release 2 (11.2) Supported Versions https://support.oracle.com/epmos/faces/ui/km/DocumentDisplay.jspx?id=888828.1
– Exadata Critical Issues https://support.oracle.com/epmos/faces/ui/km/DocumentDisplay.jspx?id=1270094.1
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 47
Graphic Section Divider