Page 1
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 1
19.04.2007
© 2006 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice
High Availability Solutions with Oracle on HP-UX
Rebecca Schlecht (HP)
EMEA HP/Oracle Cooperative Technology Centre (CTC)
DECUS IT - Symposium 2007, CCN CongressCenter Nürnberg, 19.04.2007
Slide 2
HP/Oracle Cooperative Technology Center
• Located at HP in Germany, France & Oracle UK
• 10 HP & Oracle employees in one team
• Delivering services to HP, Oracle & Partners
• Founded in spring 1994
− Technical pre-sales assistance/consulting for partners & customers
• customer workshops/demos, proof of concept projects, benchmarks, RFx contributions, architecture consulting, complex sizing and configuration assistance, customer presentations, ….)
− Know-How Transfer & Country Enablement
− Evaluation and tests of new products/features
− Technology transfer to and from US labs
Page 2
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 2
19.04.2007
Slide 3
Agenda
• HA Offering with Oracle & HP-UX
−Serviceguard
−Oracle RAC on HP-UX
• RAW, CFS, ASM
• Disaster Tolerant Solutions with Oracle RAC
• Data Guard 10gR2
Slide 4
High Availability
Page 3
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 3
19.04.2007
Slide 5
Unplanned
Downtime
Database
Maintenance
System
Maintenance
Human Error
Data Failure &
Disaster
System Failure
Online Redefinition,
Partitioning, Parallel SQL
Dynamic reconfiguration
(Patches/Drivers) + Rolling
maintenance (with SG)
Flashback, LogMiner
Data Guard, Storage Snapshots
Storage Replication (BC/CA), Data
Guard, RMAN + Data Protector
integration ...
HP Serviceguard, Oracle RAC
HP & Oracle handle all causes of downtime
Planned
DowntimeOracle Software
Rollable Patches, Data Guard
(Logical Standby), Streams
Slide 6
Cold Failover with HP Serviceguard
• Non-Shared Database
• Provides 16 node failover solution
• Applications/Users connect to Floating Virtual IP address
• Scalability of cluster is limited to scalability of one server
• Cold failover is slow, as many time consuming tasks must be performed as part of failover
− moving and mounting logical volumes
− starting the oracle instance
− opening the data files
• After failover:
− Start of Oracle instance
− All sessions must re-establish connections to the database
Application 1 Application 2
FC Switch
SAN
Clients
Ora Pkg
Virtual IP
Ora Pkg
Virtual IP
Page 4
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 4
19.04.2007
Slide 7
Workload Management with HP Serviceguard• Balance workload after a
node failure
• Minimize impact on remaining nodes
Node 4
Pkg C
Pkg H
Pkg I
Node 2
Pkg A
Pkg D
Pkg E
Node 3
Pkg B
Pkg F
Pkg G
If Node 1 fails...
Node 1
Pkg A
Pkg B
Pkg C
Slide 8
Oracle Real Application Clusters
Application
Cluster
Database
• All systems are active and access the same database
• Increase scalability beyond one system
• Much faster failover and switchover
• Can pre-connect users to the secondary instance for fast restart without logon
• Load can be distributed
• Caches are warm on the secondary instance – better performance after failure
• Requires raw devices, CFS or ASM
• Concurrent access from every node = “scale out”
• More nodes = higher availability
Page 5
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 5
19.04.2007
Slide 9
shared storage
cluster
interconnect
Oracle
Cache Fusion
S3 S4 Sn
more nodes = higher
availability
Operating System
Oracle RAC
instance 1
concurrent access from every node = “scale out”
Oracle RAC10g ArchitectureServer 1 Server 2
redo log instance 1 …
redo log instance 3 …
control files
database files
ocr
voting
Operating System
Oracle Clusterware
Oracle RAC
instance 2
opt: Vendor
Clusterware
opt: CVM / CFS
Optional:
quorum
Oracle Clusterware
opt: Vendor
Clusterware
opt: CVM / CFS
Slide 10
Oracle Cluster Synchronization Services (CSS)Node Monitoring & Membership
Hardware / OS Kernel
Oracle Cluster Ready Services (CRS)HA Framework
virtual IP addresses database instances processes
140.84.20.02
140.84.20.01
140.84.20.03SQL Cache
Large Pool
Buffer Cache
Java Pool
SGA Pool
SQL Cache
Large Pool
Buffer Cache
Java Pool
SGA Pool
Oracle Clusterware Architecture
Page 6
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 6
19.04.2007
Slide 11
Oracle RAC10g on HP-UX
RAC 10g
SG/SGeRAC & Oracle
ClusterwareOracle Clusterware Only
SLVM CVM
RAW ASM RAW CFS
ASM
Slide 12
Why HP SG / SGeRAC together with Oracle Clusterware
• HP Serviceguard as proven cluster layer with 150,000+ licenses
• HP Serviceguard is integrated with the HP Virtual Server Environment
• Continue to use HP Serviceguard to make your complete cluster high available including 3rd party applications
• HPs Enterprise Cluster Master Toolkits (ECMT) to rapidly implement and deploy applications under the control of HP Serviceguard
• Increase network reliability for all components including the critical private interconnect.
• Integration with HP Extended Cluster (aka Campus Cluster) and HP ContinentalClusters.
• Make use of a Cluster File System
• Make use of HP's shared logical volumes
• Co-existence of Oracle9i RAC and RAC10g on the same cluster
Page 7
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 7
19.04.2007
Slide 13
Integration SGeRAC & Oracle Clusterware
• In a combined SGeRAC & Oracle Clusterware environment− SGeRAC is responsible for
• Cluster membership info provided to Oracle Clusterware CSS (Cluster Synchronization Service) daemon
• Management of SGeRAC provided storage options (SLVM, CVM, CFS)
• Startup of Oracle Clusterware within an HP Serviceguard package (required 10.1.0.4).
− Oracle Clusterware CRS (Cluster Ready Services) is responsible for• Management of database and associated resources (instances,
services, VIPs, listeners, ..)
• Management of Oracle ASM
Slide 14
HP Serviceguard Extension for RAC (SGeRAC)
• Support for 9i RAC & RAC10g
• Cluster management is provided with SG
• Shared Logical Volume Manager features are enabled with SGeRAC
• Support for 16 nodes with SLVM
• SGeRAC runs on top of SG starting with 11.15
• Support for SG Extension for Fast Failover
• Support for Rolling Upgrade starting with 11.15
• SGeRAC does not support mixed clusters yet. All nodes must either be
Itanium2 or PA-RISC, but not both in the same cluster.
• Online Node Reconfiguration Support (SNOR)
• Online Node Addition Support
Page 8
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 8
19.04.2007
Slide 15
Oracle RAC10g with HP CFS for RAC
• One of HP’s new Serviceguard Storage Management Suite Bundles
• Includes the newest version of VRTS software 4.1 and SG/SGeRAC 11.17
• Provides consistent support for both Oracle 9i and 10gR2 environments, for both single-instance Oracle and RAC
• Applications and the database are managed the same way
• Provides support for 8-node clusters with SG CFS for RAC
• Ability to manage the various db files with normal UNIX commands
• CFS can be used for Oracle data files and for Oracle binaries, config files, and logs
• Ability to use all normal backup utilities with db files on a CFS
• Support for local cluster and Extended Cluster
RAC 10g
SG/SGeRAC & Oracle Clusterware
CVM
CFS
Slide 16
Serviceguard Cluster File
System
Serviceguard Cluster File
System for Oracle
T2771BA T2773BA
T2775BA T2776BA
Serviceguard
Storage
Management
Serviceguard
Storage
Management
Premium
T2772BA T2774BA
Serviceguard
Storage
Management for
Oracle
Serviceguard
Storage
Management for
Oracle Premium
Serviceguard Cluster File
System for RAC
T2777BA
HP Serviceguard Storage Management Suite
Page 9
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 9
19.04.2007
Slide 17
HP Serviceguard Storage Management Suite
Volume Management and File System
Online storage reconfiguration, online volume and file system creation and resizing, and database tablespace growth
Hot relocation of failed redundant storage
Cluster File System A CFS provides excellent I/O performance and simplifies the installation and ongoing management of a RAC database
Oracle Disk Manager (ODM) Allows near raw performance within a file system
Flashsnap Let DBAs easily establish database clone, a duplicate database on a secondary host for off-host processing
Quicklog Enhances file system performance by placing the file system log on a separate device
Dynamic Storage Tiers (formerly QoSS)
Allows single FS to span two tiers of disk (FC & ATA)
“Online archiving” solution
Use the most cost effective storage for needs
Portable Data Containers (PDCs)
Common on-disk data format across OSes
Allows for easier OS migration (i.e. Solaris � HP-UX)
Dynamic Multi-Pathing (DMP)
Allows for redundant paths to storage to be utilized, protecting against network outages
Storage Mapping Quickly diagnose performance bottlenecks from table space down to disk
Features Benefits
Slide 18
Oracle RAC10g on HP-UX with Oracle Clusterware only and ASM
RAC 10g
Oracle Clusterware Only
ASM
• The Oracle Clusterware to provide RAC cluster membership
• Oracle ASM for storage management
• High availability for other applications running on same cluster can be implemented with RAC10gR2 Cluster API
• CRS monitors the virtual IP address (VIP) and not other network interfaces. Therefore, additional software necessary to provide HA features for private interconnect (e.g. HP APA)
• No shared logical volume manager supported for Oracle files (OCR, Voting, data files, …)
• Supported with Oracle RAC 10gR1+
Page 10
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 10
19.04.2007
Slide 19
The ASM Operational Stack
Disks
Logical Vol
File System
0010 0010 0010 0010 00100010 0010 0010 0010 0010Files
Tablespace
Tables
Without ASM
Networked Storage (SAN, NAS, DAS)
Automatic
Storage
Management
Disk Group
Logical Vol
File System
File Names
Tablespace
Tables
ASM
Slide 20
ASM – How it Works
Automatic Storage Management
• Partitions total disk space into uniform sized megabyte units
• Units assigned to a file are managed and tracked by ASM
• Automatic online rebalance whenever storage configuration changes
• Only Oracle data such as data file, redo logs, dump sets, control files, spfile, archive logs, flashback logs
• Stripe And Mirror Everything (SAME)
− Striping across all disks in the disk group
− Integrated mirroring (no, double or triple mirroring)
Page 11
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 11
19.04.2007
Slide 21
Oracle ASM on top of RAW Disk
Slide 22
Oracle RAC10g on HP-UX with SG/SGeRAC & ASM over SLVM
• Prior to SGeRAC 11.17, ASM is NOT supported in SG/SGeRAC configurations
• Only support for RAC 10g R2:
− no support for RAC10g R1
− no support for single instance
− no support for Extended SG Cluster Solutions
− Each ASM disk group member must be a SLVM raw logical volume
• Up to 16 nodes in cluster
• SLVM provides active/standby multi-pathing support for RAC and single instance (“pvlinks”)
• ASM-over-SLVM enables the HP-UX devices used for disk group members to have the same names on all nodes, easing ASM configuration.
RAC 10g
SG/SGeRAC & Oracle
Clusterware
SLVM
ASM
Page 12
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 12
19.04.2007
Slide 23
Oracle ASM on top of Shared LVM
• Use of Alternate Paths, or pvlinks
• Alternate links, or pvlinks, are multiple paths to the same physical disk.
− # vgcreate <name> <primary path> <secondary path>
• Used for redundancy, in case of channel failure, alternate links will automatically change paths to a disk in the event of a card failure, increasing availability.
• No load balancing across the two paths. Therefore you typically configure the primary path of one disk over controller1 and the second disk with controller2.
− # vgcreate /dev/vg1 /dev/dsk/c4t0d0 /dev/dsk/c5t0d0
− # vgextend /dev/vg1 /dev/dsk/c5t0d0 /dev/dsk/c4t0d0
Slide 24
Configuring SLVM Volume Groups for ASM Disk Groups
Physical
Volumes (PVs)
DB Instances ASM Instance
Volume
Group
Logical Volumes
ASM Disk
Groups
DG1 DG2
• Organize the disks to be used by ASM into LVM volume groups (VGs)
• Ensure that there are multiple paths to each disk, by configuring PV Links or disk level multipathing
• For each physical volume (PV), configure a logical volume (LV) using up all available space on that PV
• On each LV, set an I/O timeout equal to pvtimeout*# of PV Links
• Export the VG across the cluster and mark it shared
• Do not use mirroring, striping, concatenation or splitting of PVs at LVM level
− ASM manages these functions
Page 13
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 13
19.04.2007
Slide 25
Oracle RAC10g on HP-UX with SG/SGeRAC & RAW
• Oracle Clusterware running integrated on top of SG/SGeRAC 11.16+
• Cluster Membership is derived from SG/SGeRAC
• SG Package Manager to provide high availability of other applications
• Provides high availability for all networks
• Integration with HP Extended Cluster solutions
• Support for shared logical volume manager (OCR, Voting, data files, …)
• Optimal performance
• Lacks file system capabilities
• Slightly more complex to handle
RAC 10g
SG/SGeRAC & Oracle
Clusterware
SLVM CVM
RAW RAW
Slide 26
RAC10g on HP-UX: What to choose?
Pro’s Con’s
RAC10g with HP SG/SGeRAC ...
+ SG/SGeRAC provides mature and proven cluster layer
+ SG Package Manager to provide HA for other applications
+ provides high availability for all networks
+ integration with HP Extended Cluster
– additional software licenses for SG/SGeRAC
– no support for heterogeneous environments (HP-UX only)
RAC10g withHP SG/SGeRAC
…and CFS
+ best management option for all kind of files
+ ability to use all normal file system utilities
+ optimal performance with ODM driver
+ includes many desirable file system + volume management features such as Flashsnap, Dynamic multipathing, …
– supported with 9i and 10g
– additional software licenses for SG CFS bundle
– support from multiple vendor
Page 14
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 14
19.04.2007
Slide 27
RAC10g on HP-UX: What to choose?
Pro’s Con’s
RAC10g with HP SG/SGeRAC
.... and RAW
+ optimal performance
+ no additional software licenses for volume manager / file systems
+ supported for 9i + 10g
– no file system capabilities
– likely more complex to handle
RAC10g withHP SG/SGeRAC
... and ASM over SLVM
+ no additional licenses for volume manager / file systems
+ easy management with ASM
+ optimal performance
+ SLVM provides active/standby multi-pathing support for RAC and single instance (“pvlinks”)
– no support before 10gR2
– no support for single instance
– no support for Extended Cluster
RAC10g with Oracle Clusterware Only and ASM
+ no additional licenses for - Volume manager - File systems
- Cluster Manager
+ easy support model (all from Oracle)
– no HA for networks
-– ASM lacks multi-pathing capabilities
– only supported with 10g
Slide 28
RAC Interconnect Support
Platform 1GbB (IP)
10GbB (IP)
IP over Infiniband
RDS over Infiniband
Hyperfabric (HMP & IP)
10g RAC HP-UX Yes Yes Yes Planned for mid this year
No
10g RAC Linux Yes Yes Yes Yes (10.2.0.3 with Silverstorm)
No
9i RAC HP-UX Yes Yes No No Yes
9i RAC Linux Yes Yes No No No
!! Crossover Cables are not supported by Oracle as an interconnect with 9iRAC/10gRAC on any platform!
Page 15
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 15
19.04.2007
Slide 29
Slide 30
Disaster Tolerant Solutions with Oracle RAC
Page 16
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 16
19.04.2007
Slide 31
Extended Distance SGeRAC Cluster
• Single Cluster over two data centers -Active/Active
• Disaster tolerance as servers and storage reside in two separate data centers
• Automatic failover to second data center
• Software mirroring
− 2 nodes supported with RAW/SLVM and MirrorDisk/UX
− 8 nodes supported with CFS/CVM and RAW/CVM up to 10km; 2 nodes up to 100km
• Support for Oracle9i & 10g
Data Center 1 Data Center 2
Storage Storage
RAC
Node A
SLVM/CVM
Node B
SLVM/CVM
Support for single-instance
& RAC
Slide 32
MetroCluster / Cluster Extension XPwith Oracle
• Single Cluster over two data centers; Active/Passive
• Disaster tolerance as servers and storage reside in two separate data centers
• Rapid, automatic site recovery without human intervention
• Storage Hardware Mirroring with XP CA, EVA CA or EMC SRDF
• Separate arbitrator for split brain situations
• system connected to mirror has read only access
Data Center 1 Data Center 2
XP/EMC XP/EMC
MC/SG
Node A Node B
Node CArbitrator
CA / SRDF
Only supported with Oracle single instance
Page 17
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 17
19.04.2007
Slide 33
HP Continentalclusters & Oracle RAC
• Single Clusters in separate data centers – Active/Passive
• Support up to 32 nodes per Continentalclusters
• Supported with Oracle single instance and Oracle RAC
• „Push-Button“ Failover
• Support for RAC 10g introduced in December 2005 with SG/SGeRAC 11.17
• Supported with RAW/SLVM, CFS/CVM
Active
DB
Primary Cluster Recovery Cluster
Instanc
e1
Instanc
e2
XP CA
Active
DB
Primary Cluster Recovery Cluster
Instanc
e1
Instanc
e2
Instanc
e1
Instanc
e2
XP CA
Before Failure:
After Failure:
Slide 34
HP Continentalclusters & Oracle RACGerman Customer Example
PU SH
REA DY
ALA RM
ME SSA GE
h p S to r a g eW or k s x p 12 0 0 0 d i sk ar r ay
h p S to r a g eW or k s x p 12 0 0 0 d i sk ar r ayh p S to r a g eW or k s x p 12 0 0 0 d i sk ar r ay
PUSH
REA DY
ALA RM
ME SSA GE
h p S to r a g eW or k s x p 12 0 0 0 d i sk ar r ay
h p S to r a g eW or k s x p 12 0 0 0 d i sk ar r ayh p S to r a g eW or k s x p 12 0 0 0 d i sk ar r ay
C abin et Num be r
HK P
48 V
GS P
A tten tion
R em ote
RAC11
Apps
Server
C abin et Num be r
HK P
48 V
GS P
A tten tion
R em ote
RAC22
Apps
Server
C ab ine t N um be r
H KP
4 8V
G SP
Att enti on
Re mo te
RAC12
Apps
Server
C abin et Num be r
HK P
48 V
GS P
A tten tion
R em ote
RAC21
Apps
Server
DB1
DB2’
DB1’
DB2
ContinentalClusters
local
SGeRAC cluster
local
SGeRAC cluster
RAC11 & RAC12 access DB1RAC21 & RAC22 access DB2
Intercluster
Monitoring
bi-directional HP CA XP/EVAsynch. or async.
Page 18
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 18
19.04.2007
Slide 35
Slide 36
Page 19
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 19
19.04.2007
Slide 37
Oracle Data Guard
Slide 38
• Replica of Primary database
• As primary database is modified, changes are propagated to standby databases
• Primary database is open and active. Standby database is either in recovery or open read-only
• If something goes wrong with primary, standby can be activated
What is a Standby Database ?
Primary Site
PrimaryDatabase
Standby Site
StandbyDatabase
Log Data
Page 20
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 20
19.04.2007
Oracle Data Guardphysical vs. logical standby
Broker
Sync/AsyncProduction Database
Network
Physical Standby Database
OptionalDelay
DIGITAL DATA STORAGE
DIGITAL DATA STORAGE
Backup
ApplyPhysical
Redo
Logical Standby Database
SQL ApplySync/Async
TransformRedo to
SQL with optional delay
ContinuouslyOpen for Reports
AdditionalIndexes & MVs
Slide 40
Data Guard Protection Modes
• Maximum Protection− Highest Level of data protection with zero data loss− Synchronous LGWR
− If standby is unavailable, processing stops at primary
• Maximum Availability− Zero data loss
− Synchronous LGWR
− If standby is unavailable, processing continues at primary. When fault corrected, standby db gets automatically resynchronized with primary db.
• Maximum Performance− Async redo data shipment to the standby database − Little effect on primary database performance
Page 21
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 21
19.04.2007
Slide 41
• Flashback DB removes the need to delay application of logs
• Flashback DB removes the need to reinstantiate primary after failover
• Real-time apply enables real-time reporting on standby
Real Time
Apply
No Delay!
Real Time
Reporting
Flashback
LogFlashback
Log
Primary: No reinstantiation after failover!
Redo
Shipment
Primary Database Standby Database
Enhanced Disaster Recovery with Flashback Database
Slide 42
Oracle Data Guard Fast-Start Failover
• Fast-Start Failover is an Oracle Data Guard 10gR2 feature that automatically fails over to a synchronized standby db in event of loss of production db without requiring any manual steps to invoke the failover.
• Requires Maximum Availability protection mode (LGWR SYNC Redo Transmission).
• Flashback Database and Flash Recovery Area must be enabled on both primary and standby db.
• After failover completes, the Broker automatically reinstates the old primary database as a new standby database.
• Configuration with Data Guard Broker that provides centralized management.
Observer
Primary Database Standby Database
Page 22
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 22
19.04.2007
Slide 43
Don’t miss …
Slide 44
Page 23
Copyright © 2006 HP corporate presentation. All rights reserved.
IT-Symposium 2007
www.hp-user-society.de 23
19.04.2007
Slide 45