SAP on RAC Update Kurt Broeg Jan Klokkers Principal Member of Senior Director Technical Staff SAP Development Server Technologies Oracle Corporation
Oct 22, 2014
SAP on RAC Update
Kurt Broeg Jan Klokkers Principal Member of Senior DirectorTechnical Staff
SAP DevelopmentServer Technologies
Oracle Corporation
Topics
� Introduction� Certification Status� Mandatory SAP Requirements for RAC� SAP Certified Technology Stack for RAC 9.2� Necessary Technology Stack for SAP with RAC 10.2� Introduction of Net Services� SAPCTL
Introduction
� More than 18 years of SAP – Oracle cooperation� More than 12,000 SAP customers use Oracle� 2 out of 3 SAP installations run on Oracle Database� Joint SAP & Oracle development/platform team in
Walldorf� Joint support teams in Rot (Germany) & US� SAP & Oracle Partner/Presales teams in Walldorf
Certification Status –Supported Platforms of SAP for Oracle
� IBM AIX5L� HP-UX PA-RISC� HP-UX IA64� HP Tru64� Linux x86� Linux x64� Linux IA64� Linux Power
� Solaris Sparc-64� Solaris x64 (AMD64)� Windows x86� Windows x64� Windows IA64
Certification Status – Supported Oracle Versions
� Oracle 9.2.x– Single Instance and RAC– SAP versions 4.6C, 4.7, ERP2004
� Oracle 10.1.0.x (x>=4)– Only Single Instance– Only for SAP ERP2005
� Oracle 10.2.0.x (x>=2)– Single Instance available– RAC 10.2 availability planned for Q4CY2006– SAP versions 4.6C, 4.7, ERP2004 and ERP2005
Mandatory SAP Requirements for RAC
� Shared Oracle Home– No support for local Oracle Homes
� Database on Shared Filesystem– No support for Raw Devices
– No support for ASM (planned for Oracle11)
� Shared Archive Destination on Shared Filesystem� Shared SAP Directories on Shared Filesystem
– saparch, sapbackup, sapreorg, saptrace, sapcheck
Certified RAC 9.2 Configurations for SAP
Oracle Clusterware + OCFS 9.2.0.7/8 2003 Enterprise ServerWindows
Veritas Foundation Suite 4.0/5 or SUN Cluster 3.1+ QFS
8, 9, 10Solaris (SUN)
Veritas Foundation Suite 4.0/5 or PrimeCluster + Net App Filer
8, 9, 10Solaris (Fujitsu Siemens)
Oracle Clusterware andOCFS2 1.2.x or NetApp NFS
SLES9, RHEL4Linux
Oracle Clusterware and
OCFS 1.0.14 or NetApp NFS
SLES8, RHEL3Linux
HACMP 4.5/5.1 + GPFS 2.2/2.35.2/3IBM AIX
TruCluster Server 5.15.1bHP Tru64
ServiceGuard with SGCFS 11.23HP-UX
Cluster Version and CFS TypeOS Version
Required RAC 10.2 Configurations SAP
Oracle Clusterware + OCFS 10.22003 Enterprise ServerWindows
Veritas Foundation Suite 4.0/5 or SUN Cluster 3.1+ QFS or
Oracle Clusterware + NetApp NFS
9, 10Solaris (SUN)
Veritas Foundation Suite 4.0/5 or PrimeCluster + Net App Filer orOracle Clusterware + NetApp NFS
9, 10Solaris (Fujitsu Siemens)
Oracle Clusterware and OCFS2 1.2.x or NetApp NFS
SLES9, RHEL4Linux
HACMP 5L + GPFS 2.3 or
Oracle Clusterware + GPFS 2.3
5.2/3IBM AIX
TruCluster Server 5.15.1bHP Tru64
ServiceGuard with SGCFS 11.23HP-UX
Cluster Version and CFS TypeOS Version
Q U E S T I O N SQ U E S T I O N SA N S W E R SA N S W E R S
Providing High Availability for SAP Resources
Abstract
Oracle Clusterware is the component that’s responsib le for restarting the Oracle Real Application Cluster Inst ance and Listeners on process failure & relocating Database Virtual IP’s on Node failure. A key new feature for Oracle Database 10g Release 2 – Oracle Clusterware is now made availa ble for customers and partners to protect 3rd Party Applica tions.
Especially for SAP, Oracle provides a tool sapctl which uses the capabilities of Oracle Clusterware to support th e SAP Replicated Enqueue feature.
This presentation will cover the implementation and use of sapctl in SAP HA - systems.
Concept of SAP Enqueue Replication
The SAP Enqueue Service provides distributed lock m anagement forSAP R/3 instances. The SAP Enqueue Service must be available whenever an SAP installation uses more than one SAP R/3 instance.
SAP uses an active/passive approach to enable high availability for the Enqueue Service. There is exactly one Enque ue Service running in the SAP system. The Replication Service run on the node on which the Enqueue Service runs in the event of a failure or planned outage.
The Replication Service maintains a copy of the Enq ueue Service ´s lock table; this copy is kept in a shared memory se gment during runtime.
The node that supports the Enqueue Service also host s a virtual IP address (VIP) for communication. Whenever the active node fails or is shut down, the VIP and the Enqueue Service fail ove r to the next node.
Control flow within SAPCTL to support SAP Enqueue Replication
Enqueue
Server
Lock
Table
Replication Server
Lock Replica Table
TCP Connection
Lock State
Transfer
0. Normal Operation, CRS checks healthiness of resources on all hosts in the cluster
VIP
Host 1 Host 2
Control flow within SAPCTL to support SAP Enqueue Replication
Enqueue
Server
Lock
Table
Replication Server
Lock Replica Table
1. CRS detects a failure of the node or in the Enqueue Server
VIP
Host 1 Host 2
Crash
Control flow within SAPCTL to support SAP Enqueue Replication
Replication Server
Lock Replica Table
2. CRS switches the VIP to the node where Replication Server is running
Host 1 Host 2
VIP
Control flow within SAPCTL to support SAP Enqueue Replication
Replication Server
Lock Replica Table
3. CRS starts a new Enqueue Server on the node supporting the VIP
Host 1 Host 2
VIP
Enqueue
Server
Lock
Table
Control flow within SAPCTL to support SAP Enqueue Replication
Replication Server
Lock Replica Table
4. Enqueue Server reads data from Lock Replica Table
Host 1 Host 2
VIP
Enqueue
Server
Lock
Table
fill Enq. Table with
replica Data
Control flow within SAPCTL to support SAP Enqueue Replication
5. CRS stops Replication Server after Lock Table is rebuild from Lock Replica Table
Host 1 Host 2
VIP
Enqueue
Server
Lock
Table
Control flow within SAPCTL to support SAP Enqueue Replication
6. CRS starts Replication Server on another available node in the cluster
Host 1 Host 2
VIP
Enqueue
Server
Lock
Table
Host N
Replication Server
Lock Replica Table
Control flow within SAPCTL to support SAP Enqueue Replication
7. Back to normal operation, CRS checks healthiness of resources on all hosts in the cluster
Host 1 Host 2
VIP
Enqueue
Server
Lock
Table
Host N
Replication Server
Lock Replica Table
Lock State
Transfer
Connect
Possible Configuration
� Oracle Clusterware HA Agent for SAP is protecting SAP Enqueue Server and SAP Enqueue Replication Server
� It is also providing resources to the RAC database running on nodes 1 & 2
listener
SAP Enqueue Server
ORA_CRS_HOME
ORACLE_HOME
listener
SAP Enqueue Repl.
cluster database
ORA_CRS_HOME
ORACLE_HOME
ORA_CRS_HOME
listener
database
ORA_CRS_HOME
ORACLE_HOME
SAP Enqueue Repl.
cluster database
The Oracle Clusterware Framework
� Calls SAP HA - agent� agent will be invoked by Oracle Clusterware
with one of three commands– start– stop– check
� Oracle Cluster Registry– Stores information relating to how Oracle
Clusterware interacts and manages the application
Oracle ClusterwareFramework Architecture
AgentFramework
Please Start
How are you ?
Please Stop
‘check’
‘start’
‘stop’
How the HA Agent for SAP is called by the Framework
The ‘start’ commandWould invoke the sapstart command
startsap r3 ASCSstartsap r3 ENR
The ‘check’ commandThere are a number of things that could be checked
Uses ensmon to obtain the status of the replicationBut: Does not check the message server
The ‘stop’ commandWould invoke the stopsap command
stopsap r3 ASCSstopsap r3 ENR
‘start’
‘stop’
‘check’
� sapctl create - registers resources � sapctl config - shows configuration� sapctl status - shows status of resources� sapctl start - starts resources� sapctl stop - stops resources� sapctl update - modfy configuration � sapctl remove - delete configuration
Overview: SAPCTL commands for use with SAP R/3 EnqueueReplication
� sapctl create -sapsid C11 <- name of the SAP System -if eth0 <- network interface-nm 255.255.254.0 <- netmask-ip 140.86.242.63 <- IP address VIP-u c11adm <- SAP admin-n “node1 node2” <- list of nodes-s /saphome/c11adm <- $HOME of SAP admin-e ASCS01 <- Central Instance-r ENR01 <- Enqueue replication
SAPCTL command create
� sapctl configshows list of protected SAP systems
� sapctl config enqueue –sapsid C11shows configuration of Central Instance
� sapctl config replication –sapsid C11shows configuration of Replication Server
� sapctl config all –sapsid C11shows complete configuration for SAP system
SAPCTL command config
� sapctl status enqueue –sapsid C11shows status of Central Instance
� sapctl status replication –sapsid C11shows status of Replication Server
� sapctl status all –sapsid C11shows status of all SAP resources
SAPCTL command status
� sapctl start enqueue –sapsid C11starts SAP Central Instance
� sapctl start replication –sapsid C11starts SAP Replication Server
� sapctl start all –sapsid C11starts all SAP resources
SAPCTL command start
� sapctl stop enqueue –sapsid C11stops SAP Central Instance
� sapctl stop replication –sapsid C11stops SAP Replication Server
� sapctl stop all –sapsid C11stops all SAP resources
SAPCTL command stop
� sapctl update -sapsid C11 <- name of the SAP System
-ip 140.86.242.63 <- IP address VIP
-a “node1 node2” <- list of nodes to add-d “node1 node2” <- list of nodes to delete
SAPCTL command update
� sapctl remove all –sapsid C11removal of all SAP resources
- only remove of all SAP resources possible- `root` permission required
SAPCTL command remove
Commonly used scripts startsap and stopsap trigger the start / stop
of CRS resources as well. Consider this as either a very convenient
feature or a big advantage, as other tools or scripts relying on thesecommands still work.
startsap r3 ASCS01 - starts Enqueue and Message Server
stopsap r3 ASCS01 - stops Enqueue and Message Server
startsap r3 ENR01 - starts Replication Server stopsap r3 ENR01 - stops Replication Server
Restrictions apply: User must use this command on the right node!
SAP start/stop commands still usable
startsap script:
• add instance type ENR / ASCS / SCS• add service type ER • call wrapper to notify Oracle Clusterware in function
start_instance if service is “CS” or “ER” :
sap2crs start \$SAPSYSTEMNAME \$SERVICE \$INSTANCE
SAP changes required to use SAPCTL
stopsap script:
• add instance type ENR / ASCS / SCS• add service type ER • call wrapper to notify Oracle Clusterware in function
stop_instance if service is “CS” or “ER” :
sap2crs stop \$SAPSYSTEMNAME \$SERVICE \$INSTANCE
SAP changes required to use SAPCTL
• Instance profile Standalone Enqueue Server (ENQ + MSG)
• Instance profile Enqueue Replication Server (ENR)enque/process_location=LOCALenque/server/replication=trueenque/server/internal_replication= true
• Start profile Standalone Enqueue Server• Start profile Enqueue replication Server
As recommended by SAP HA documentation
SAP changes required to use SAP HA
• All scripts exist multiple times: START_ASCS01_node-1, C11_ASCS01_node-1START_ASCS01_node-2, C11_ASCS01_node-2
…..START_ASCS01_node-n, C11_ASCS01_node-n
The hostname is not virtualized by Oracle Clusterware!
SAP changes required to use SAP HA
Demonstration
Further Information available:
� Oracle Whitepapers:
1. Technical White Paper: Providing High Availability for SAP Resources
2. SAP NetWeaver with Oracle 10gR2 RAC (UNIX & Linux)
Oracle Location:http://www.oracle.com/sap -> Oracle on RAC section
SAP Location:http://service.sap.com/dbaora ->Oracle Database
Administration -> Media Library -> Oracle RAC
Q U E S T I O N SQ U E S T I O N SA N S W E R SA N S W E R S