IBM Software Group © 2008 IBM Corporation Business Continuity Advances with IBM Tivoli System Automation Family Dr. Norbert Lenz [email protected] IBM Software Group October 2008
Jan 05, 2016
IBM Software Group
© 2008 IBM Corporation
Business Continuity Advances with IBM Tivoli System Automation Family
Dr. Norbert Lenz [email protected]
IBM Software Group
October 2008
IBM Software Group | Tivoli software
© 2008 IBM Corporation2 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Copyright and Trademarks
© Copyright IBM Corporation 2008
The following names are trademarks of the IBM Corp. in USA and/or other countries and may be used throughout this presentation:
AIX, CICS, DB2, eLiza, Enterprise Storage Server, ESCON, FICON, FlashCopy, GDPS, HyperSwap, IBM, IBM eServer, IMS, MVS/ESA, MQSeries, NetView, OMEGAMON, Parallel Sysplex, RMF, RACF, S/390, Syspex Timer, Tivoli, VTAM, WebSphere, z/OS, z/VM, System z, System p, System i
Other company, product and service names may be trademarks or service marks of others.
IBM Software Group | Tivoli software
© 2008 IBM Corporation3 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Agenda
IBM Tivoli System Automation – Product family and basics
IBM Tivoli System Automation for Multiplatforms
GDPS/PPRC Multi Platform Resiliency for System z (xDR)
IBM Tivoli System Automation Application Manager
Business Continuity Process Manager
IBM Software Group | Tivoli software
© 2008 IBM Corporation4 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Business Continuity Issues IT challenges
• Downtime unaffordable– Planned and unplanned
• Heterogeneous environments• Complexity
Customer pressures• Application availability • Operations complexity and costs• Automation implementation and
maintenance costs• Education requirements related• Rapid change of I/T infrastructure
• Loss of business• Loss of customers – the competition is just a mouse click away• Loss of credibility, brand image and stock value
Reasons for planned downtime• Maintenance• Tests
Reasons for unplanned downtime• Operator errors• Application failures• Environmental failures
– OS failures– HW failures– …– Disasters
40%Operations
Errors
40%Application
Failure
20% Environmental
Factors, HW, OS, Power,
Disasters
20% Environmental
Factors, HW, OS, Power,
Disasters
Source: Gartner Group, 2007
IBM Software Group | Tivoli software
© 2008 IBM Corporation5 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Customer Value of Business Continuity
BC reduces the risk of a business interruption•Ensures Continuous Availability (CA) of applications
–Continuous Operation (CO)
Minimizes impact of planned outages–High Availability (HA)
Minimizes impact of unplanned outages
BC helps to stay in business when a severe interruption occurs
•Supports Disaster Recovery (DR)
HAHA
Site Mgmt
Data Mgmt Data Mgmt
Site Mgmt
DR DR
Replication
IBM Software Group | Tivoli software
© 2008 IBM Corporation6 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Vision For a Business Continuity SolutionBusiness Continuity…
Provides both Continuous Availability and Disaster Recovery for heterogeneous distributed IT business applications
• "Recover my business rather than my platform technology“
Ensures successful recovery via automated processes
• Pre-canned, pre-tested
• Can be handled by less-skilled operators
Allows management on the “business impact” level
• Granular
• Customer flexibility on desired RTO, RPO quality of service
Leverages existing customer/IBM investments and technology
System z is the Business Continuity management hub
IBM Software Group | Tivoli software
© 2008 IBM Corporation7 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Tivoli System Automation EvolutionNetwork Edge/Web Application Data/Transaction
Leading the Enterprise
Step I: Componentize SA z/OS
SA z/OS
SA MP
Step II: Leverage components across platforms
Step III: Integrate operational capabilities end-to-end
Automation Engine
SA z/OS
e2eSA App Man *
e2e
ISMBCPM
Step IV: Business Continuity: Leverage automation technology in IBM Service Management
SA MP on Linuxon System z,x or p
MSCSVCS
HACMP
SA MP on AIXOr SUN Solaris
SA z/OS
SA MP on Win
SA MP on AIX
SA MP on Linux(x,p,z)
SA MP on Windows
* SA Application Manager is the new name for SA MP End to End Automation
SA MP on SUN Solaris
NewV3.1
NewV7.1
IBM Software Group | Tivoli software
© 2008 IBM Corporation8 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Business Continuity Solution Architecture
Change Management
Configuration Management
Release Management
Storage Process Manager
Clustered Applications / Middleware /
OMP / Storage Systems
End-to-End Multi-Tiered
Application and Storage System
Process Management
Process Management Infrastructure
AvailabilityProcess Manager
IBM Service Management Process Runtime InfrastructureCMDB Maximo Integration Modules
IBM Tivoli System AutomationApplication Manager
AIX / LinuxWindows / Solaris z/OS
Business ContinuityProcess Manager
New inV3.1
New V7.1
IBM Software Group | Tivoli software
© 2008 IBM Corporation9 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
System Automation Basic Principles
Coordinated
Restart
& Failover
Automation
Agent
Event
Automation
Engine
Operator
Administrator
•Restart & FailoverRules•Resource Groups•Relationships and Dependencies
Scheduler
Orchestration
mySAPPolicy
WASPolicy
CustomerPolicies
…MQPolicy
IMSfeature
CICSfeature
Operator
DB2Policy
• Capability to automatically start and stop resources with knowledge of…• Resource groups, relationships, backup resources, …
• Provide Continuous Availability of IT resources• Applications, processes, IP addresses, file systems, …
IT Manager
IBM Software Group | Tivoli software
© 2008 IBM Corporation10 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Goal-driven - Automation in a Nutshell
Request ChangesMonitor Events
Resources & Relationships & Rules
Evaluate relationship correlation rules
Apply state change
Find actions to Submit Action
Translate event to state
Translate Action
Resources
Decide
Monitor and Control
M
A
P
E
Resource DefinitionDesired Operational StatesStart/Stop SequencesGroupingFailure Actions
Automation Policy
reach goal
IBM Software Group | Tivoli software
© 2008 IBM Corporation11 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
What is the Power of an Automation Policy? Easy definition
• Pre-defined automation for common applications
• Faster time-to-value
• No script writing
– Elimination of coding errors– Easy to maintain
• Easy to build ‘business view’
– Group nesting, arbitrary abstraction level
• More efficient use of scarce
– ‘people’ resources
Consistent, reliable, automation actions • Testing abnormal condition actions is difficult and sometimes incomplete with scripts
• Policy definitions can be cloned for similar requirements elsewhere in the enterprise
• Management of entire business applications, rather than individual resources
Resources
Aggregated Resources
Business System
IBM Software Group | Tivoli software
© 2008 IBM Corporation12 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
IBM Tivoli System Automation for z/OS Proven solution for Automation of IBM z/OS Sysplex
• Best of breed mainframe automation
• More than 1000 applications in sysplex with complex dependencies
• Automated Operation and High Availability
• Grouping and Relationships described in Automation Policies
• Integration with Performance Monitoring (TEP, OMEGAMON XE, CICSPLEX SM)
CF
AgentAgent
Agent
AutomationManager
z/OS Sysplex
IBM Tivoli System Automation
for z/OS
New inV3.2
IBM Software Group | Tivoli software
© 2008 IBM Corporation13 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Agenda
IBM Tivoli System Automation – Product family and basics
IBM Tivoli System Automation for Multiplatforms
GDPS/PPRC Multi Platform Resiliency for System z (xDR)
IBM Tivoli System Automation Application Manager
Business Continuity Process Manager
IBM Software Group | Tivoli software
© 2008 IBM Corporation14 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
IBM Tivoli System Automation for Multiplatforms (SA MP) Automated Operation and High Availability of Applications on Linux, AIX and Windows
• Automates failover, stop, start, restart in place and move operations for applications
• Supports AIX, Linux on System z, x, p, i, Windows and SUN Solaris
• Monitors and controls
– Applications– Network (TCP/IP)– File Systems
• Automated Operations & High Availability with best of breed Automation Manager from IBM Tivoli SA for z/OS
• Based on Automation Policies
Linux, AIX or Windows
ResourceManager
ResourceManager
ResourceManager
AutomationManager
RSCT
IBM Tivoli System Automationfor Multiplatforms
IBM Software Group | Tivoli software
© 2008 IBM Corporation15 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
IBM Tivoli System Automation for Multiplatforms
Event: On Start of RG_WebRG_Web already Online?Yes exitNo continue
All members Offline?No Problem, exitYes continue
DB startable on node1?No goto StartNode2Web startable on node1?
No goto StartNode2
Start DB on node1
Wait until DB is OnlineStart Web on
node1
Wait until Web is OnlineSet group
status to OnlineExit
StartNode2:DB startable on node2?No goto StartNode3 or
exit Web startable on node2?
No goto StartNode3 or exit
Start DB on node2
Wait until DB is OnlineStart Web on
node2
Wait until Web is OnlineSet group
status to OnlineExit
Event: On failure of DBRunning on node1?
Stop Web on node1Wait until Web is OfflineGenerate Event: On Start
of RG_WebExit
Running on node2?Stop Web on node2
Wait until Web is OfflineGenerate Event: On Start
of RG_WebExit
Event: On failure of WebRunning on node1?Restart on node1 possible?
Yes Start Web on node1
Wait until Web is Online
ExitNo Stop DB
Wait until DB is OfflineGenerate Event:
On Start of RG_WebExit
Running on node2?Restart on node2 possible?
Yes Start Web on node2
Wait until Web is Online
ExitNo Stop DB
Wait until DB is OfflineGenerate Event:
On Start of RG_WebExit
Web
DBVIPA
DependsOnDependsOn
No programming required Simple policy with
• Resource information
• Relationships
• Groups of resources
• Incremental policy activation / deactivation
• Template Processing
Allows operators to control applications on
business level
• Frees operators from remembering application
components and relationships
New resources or systems can be added
without re-writing scripts
Automation Policy Reduces Implementation Time, Coding and Support Effort
Web
DBVIPA
IBM Software Group | Tivoli software
© 2008 IBM Corporation16 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
NodeNodeNodeNode
AppServer Inst
replicate ERS
NodeNode
ERS
ES MS
VIPA
replicate
conn
ect
ES MSVIPA
connect
Example: mySAP Enqueue Server Recovery
SA Policies: ES, MS, VIPA collocated ERS starts after ES ERS is anti-collocated to ES ES collocated to ERS if online and ES offline
SA Actions:1. Ensure correct start-up sequence /
node of ES and prereq’s
2. Ensure correct start-up sequence / node of ERS
3. Recognize Node 1 failure
4. Failover ES and prereq’s to Node 2
5. Connect SAP App Server to VIPA on Node 2
6. Wait for information transfer from ERS to ES thru shared memory
7. Move ERS to Node 3
8. Resume SAP operations
IBM Software Group | Tivoli software
© 2008 IBM Corporation17 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Pre-canned Automation Policy Templates (Linux / AIX) Data Management
•DB2 v9 (ESE, ESE DPF, HADR) (Note: SA MP is shipped with DB2 on Linux and AIX)•DB2 BCU•DB2 8.x (ESE, ESE DPF, HADR)•DB2 7.x WE, EE•Oracle 10g•Oracle 9i•Oracle 8i
Tivoli Products•Tivoli Provisioning Manager 5.1.1•ITM 6.2•CCMDB V7.1•TADDM 7.1•CCMDB V1.1.1 (this includes TADDM, WebSphere Application Server, WebSphere Portal Server, DB2, IBM HTTPS, and TDS)•Tivoli System Automation for Multiplatforms (SA MP) (note: for the End-to-End Automation Component)•Tivoli Workload Scheduler•Tivoli Storage Manager (TSM) (includes Client, Server and Admin Server)•Tivoli Enterprise Console 3.8
SAP •SAP Replicated Enqueue environment•SAP Application Server•SAPDB•(Note: SA MP is shipped with DB2 on SAP DVD and integrated in SAP‘s installation procedure)
WebSphere•WebSphere Application Server 6.0•WebSphere MQ
Shared File Systems•NFS Server•NFS Client•Samba
Groupware•Sendmail 8.11
Replication•GLVM (AIX)•Metro Mirror with DS4000
Web Servers•Apache Web Server•IBM HTTP Server
Currently under development / planned:•Tivoli Directory Server (TDS)
new
IBM Software Group | Tivoli software
© 2008 IBM Corporation18 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Weber Automotive Drives its SAP Software with Linux on POWERWeber Automotive GmbH, based in Markdorf, Germany, manufactures drive components and fiber reinforced plastic parts for the automotive industry.
To promote the high availability that Weber Automotive needs from its production systems, the company also chose to deploy IBM Tivoli System Automation for Multiplatforms (Tivoli SA). The software manages the SAP application clusters and uses policy-based self-healing technologies to analyze and fix most problems automatically – whether they occur in the application, operating system, or middleware layer.
"Tivoli SA comes with a plug'n play automation module for SAP applications, so it can deal with the majority of problems that occur in SAP software environments with minimal customization," says Günter Dürringer, Head of IT at Weber Automotive. "By understanding the relationships between application components, Tivoli SA finds and resolves issues quickly, and minimizes the need for IT staff to intervene."He adds: "With Tivoli SA, we are able to maintain very high availability for our SAP software environment, ensuring that systems remain online and accessible to users at all times."
IBM Software Group | Tivoli software
© 2008 IBM Corporation19 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
SA MP Towards a Distributed World
Linux on System z
Solarison SUN-servers
z/OS on System z
AIX and Linux on System p
HP-UXon HP-servers
Linuxon BladeCenter
Linux on iSeries
Linux on PCs
(System x and non-IBM PCs)
Windows on PCs
(System x and non-IBM PCs)
SA MP SA MP
SA MPSA MP
SA MP
SA MPSA z/OS
System z„classical“ UNIX ServersPC Systems
SA MP
New inV3.1
IBM Software Group | Tivoli software
© 2008 IBM Corporation20 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Agenda
IBM Tivoli System Automation – Product family and basics
IBM Tivoli System Automation for Multiplatforms
GDPS/PPRC Multi Platform Resiliency for System z (xDR)
IBM Tivoli System Automation Application Manager
Business Continuity Process Manager
IBM Software Group | Tivoli software
© 2008 IBM Corporation21 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
GDPS Overview Continuous Availability and Disaster Recovery solution
•GDPS/PPRC (Peer to Peer Remote Copy (PPRC) - Synchronous)
– PPRC – capable disk subsystems are supported•Multisite Sysplex (fiber distance between sites 100 km - max)
•No or limited data loss in unplanned failover - user policy
•Planned and Unplanned reconfiguration support
Disaster Recovery solution •GDPS/XRC (eXtended Remote Copy (XRC) - Asynchronous)
•Supports any distance
•Production systems in Site 1
•Limited data loss to be expected in unplanned failover
•GDPS initiates restart of production systems in Site 2
Common functions (GDPS/PPRC and GDPS/XRC)•GDPS solution extended to tape resident data
•Point-in-time copy created (FlashCopy™)
•Management of System z Operating Systems (Linux® on System z, z/VM™ , VSE/ESA™ )
1 4
3
2
ESCON
PPRC
1 4 3 2
XRC
SDM
UNIX, NTz/OS
Virtual Tape Controllers
Virtual Tape Controllers
Primary Site
Secondary Site
TCDBTMC
Catalog
TCDBTMC
Catalog
PPRC
PtP VTS
IBM Software Group | Tivoli software
© 2008 IBM Corporation22 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Re-IPL in Place If
• GDPS detects that a z/OS system is no longer active
• Auto IPL has been enabled
• The threshold of the number of IPLs in the predefined time window has not been exceeded
• No planned action is active Then
• GDPS automatically re-IPLs the system in place
• Brings it back into the Parallel Sysplex
• IBM Tivoli System Automation for z/OS (SA z/OS) restarts the workload
z/OS
Check Conditions
Re-IPL
IBM Software Group | Tivoli software
© 2008 IBM Corporation23 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Site takeover
Reset LPARs
Start workloadon site 2
When GDPS detects a complete site failure, it can offer a takeover of the workload on site 2• If additonal capacity is needed on the backup site, Capacity Backup Upgrade (CBU) can be used
CBU
Site 2Site 1
IBM Software Group | Tivoli software
© 2008 IBM Corporation24 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
HyperSwap
Site 1 Site 2
PPRCPPRC
capable disks
I/O
z/OS z/OS
I/O
When GDPS detects an disk outage, access to primary disks can be switched to secondary disks
Can swap large number of devices very fast Includes all volumes attached to the system Non-disruptive – applications keep using same logical device addresses Planned (Maintenance) and Unplanned (High Availability) Scenario
PPRC capable
disks
IBM Software Group | Tivoli software
© 2008 IBM Corporation25 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
GDPS/PPRC Multi Platform Resiliency for System z (xDR)
Business Continuity Solution for z/OS and Linux on System z on z/VM
• Based on GDPS
• Leverages existing and proven solutions
– GDPS – SA z/OS– IBM Tivoli System Automation for
Multiplatforms (SA MP)
• Provides coordinated cross platform business resiliency for OSs running on System z hardware
• Integration point of z/OS and Linux on System z
GDPS
SA MP
SA z/OS
NetView
z/OS
Linux on System z
z/VM
System z Hardware
IBM Software Group | Tivoli software
© 2008 IBM Corporation26 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
GDPS/PPRC Multi Platform Resiliency for System z (xDR)
xDR extends GDPS to support not only z/OS but also Linux on System z• Disk error detection
• Heartbeat for sanity checks
• Re-IPL in place
• Coordinated Site Takeover
• Coordinated HyperSwap
xDR is not a Product• In order to implement xDR, the following products are needed:
–GDPS
–System Automation for Multiplatforms
–Linux on System z
–z/VM, if Linux is running on z/VM
IBM Software Group | Tivoli software
© 2008 IBM Corporation27 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
GDPS/PPRC Multi Platform Resiliency for System z Scenarios Examples:
• WAS, Portlet servers running on Linux on System z under z/VM, CICS and DB2 running on z/OS sysplex• mySAP app servers running on Linux on System z under z/VM, mySAP DB server running on z/OS• Other Linux on System z applications, e.g. mail servers running on Linux on System z under z/VM• Communication Controller for Linux on System z
ECKD
GDPS K system on
z/OS
LPAR4
NetView
SA z/OS
ECKDPPRC
Site 1
CICS,
DB2 on z/OS
LPAR2
SA z/OS
NetView
mySAP
DB Serv on z/OS
LPAR3
SA z/OS
NetView
Expendable work
load
LPAR6
Expendable work
load
LPAR5
z/OS Sysplex
CBU
Site 2
Capacity Upgrade on
Demand
GDPS Site Takeover / Failover
HyperSwap
LPAR1
z/VM
S
S
S
L
A
e
A
I
P
r
N
v
M
U
App
s
P
X
IBM Software Group | Tivoli software
© 2008 IBM Corporation28 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
xDR Customer Value
Industrial strength business continuity solution for Linux on System z based on GDPS•Pretested solution, enables lower skilled operators to perform DR
High customer value for coordinated Linux on System z – z/OS business continuity•Single point of control to manage cross platform business continuity
•Coordinated HyperSwap
-E.g. because storage subsystems are used by both, Linux on System z and z/OS
•Coordinated site takeover / failover
Business continuity augments the benefits for Server Consolidation on Linux on System z
IBM Software Group | Tivoli software
© 2008 IBM Corporation29 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
CeBIT Demo: Unplanned HyperSwap Scenario
PPRC
GDPS HyperSwap
EIF
Site 2
z/OS
NetView
GDPS
Autom.
Table
SA z/OSSite 1
z/VM
Production
Linux
xDR Proxy
CLI
LOGRDR
SA MP
CPINT
Web
Online
Shop
z/OS
Production z/OS
SA z/OS
FTPD
z/OS Sysplex
ESS ESS
IBM Software Group | Tivoli software
© 2008 IBM Corporation30 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
GDPS/PPRC Multi Platform Resiliency for System z GDPS/PPRC Multi Platform Resiliency for System z (xDR)(xDR)
Coordinated near-continuous availability and DR solutionNew support: two K Systems, z/VM CSE data sharing
CF1
K1z/OS
P1z/OS
Linux z/VM P2z/OS
K2z/OS
SITE 1 SITE 2CF2
S SPL PL
L
k-sys
LP
LS
VMLinux Guests
Linux Guests
CDSCDS
LL
k-sys
Linux GuestProxy
GDPS automates
Linux startup
CBU
Planned &
Unplanned HyperSwap
Boom!
Requires Tivoli System Automation for Multiplatforms
LL
IBM Software Group | Tivoli software
© 2008 IBM Corporation31 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
GDPS/PPRC Multiplatform Resilience for System z (aka xDR) – active / active - HyperSwap Experience
z/OS PPRC Volume Pairs
PPRC Pairs per z/VM Cluster
Planned HS RESYNCH
UIT
Planned HS SUSPEND
UIT
Unplanned HyperSwap
UIT
1,200 20 40 sec 8 sec 4 sec
UIT = User Impact Time
z/VM Linux
z/OS ... z/OS
CF
z/OS ... z/OSz/VM Linux
CF
P
10-way Parallel Sysplex (CICS, WebSphere, DB2)
HyperSwap
CF Duplexing
S
Business Requirements No loss of committed data (RPO=0) Continuous data availability for z/OS and
Linux hosted by z/VM Supporting site maintenance without
application outage Coordinated disaster recovery for
heterogeneous System z applications (RTO < 30 minutes)
5 km
z/VM Cluster
z/VM Cluster
IBM Software Group | Tivoli software
© 2008 IBM Corporation32 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
GDPS/PPRC Multi Platform Resiliency for System z GDPS/PPRC Multi Platform Resiliency for System z (xDR)(xDR)
Coordinated near-continuous availability and DR solutionNew support: two K Systems, Linux swap disks mirrored with mdraid
CF1
K1z/OS
P1z/OS
Linux z/VM P2z/OS
K2z/OS
SITE 1 SITE 2CF2
S SPL PL
L
k-sys
LP
LS
Native Linux
Native Linux
CDSCDS
LL
k-sys
Native LinuxSwap
GDPS automates
Linux startup
CBU
Planned &
Unplanned HyperSwap
Boom!
Requires Tivoli System Automation for Multiplatforms
Native LinuxSwap
LL
mdraid
IBM Software Group | Tivoli software
© 2008 IBM Corporation33 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Agenda
IBM Tivoli System Automation – Product family and basics
IBM Tivoli System Automation for Multiplatforms
GDPS/PPRC Multi Platform Resiliency for System z (xDR)
IBM Tivoli System Automation Application Manager
Business Continuity Process Manager
IBM Software Group | Tivoli software
© 2008 IBM Corporation34 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Actual Application Architecture for Consumer Electronics Company
SA Application Manager is Key for Business Continuity
AutomationManager
System Automation
Manages applications and resources with complex relationships
Automated operations
High availability
Rapid error detection
Clustered environments
SA Application Manager
Cross cluster
Cross platform
IBM Software Group | Tivoli software
© 2008 IBM Corporation35 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
SA Application Manager
Enterprise Service Group
Windows Linux ClusterSUN Solaris
startsAfter
First Level Automation
End-to-End Automation LevelApp. ServerWEB Server Database
IBM Tivoli System AutomationApplication Manager
My Business Application
Customers need to automate operations for n-tier applications – Enable operators to work on End-to-End level without having Operating System -
specific knowledge – Reduce labor cost associated with maintaining availability levels– Reduce cost of downtime
– shorten repair/maintenance windows– reduce operator errors
– Based on best of breed Automation Manager and Automation Policies
IBM Software Group | Tivoli software
© 2008 IBM Corporation36 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Automation Operation ConsoleDomain Hierarchy
-SA MP Automation Clusters (Linux or AIX)-SA z/OS Automation Sysplexes (z/OS)-HACMP (AIX)-MSCS (Win)
Pageable Resource Table- Hierarchy of Automated Resources
Monitor & Control all Resources
-Name, Owner & SupportInfo-Resource State-Relationships-Start and Stop Operation
The end-to-end automation domain supports the capability to suspend automation for resources• Use this feature e.g. when applying service
IBM Software Group | Tivoli software
© 2008 IBM Corporation37 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Tivoli System Automation Product Family
IBM Tivoli System AutomationApplication Manager
IBM Tivoli System Automationfor Multiplatforms
IBM Tivoli System Automation
for z/OS
Automation Adapter
Automation Adapter
Automation Adapter
Other Clustering/HAProducts:
• IBM HACMP• MSCS• Veritas
Linux on x SeriesLinux on p SeriesLinux on z SeriesAIXWindows
SUN Solaris
z/OS
Linux on x SeriesLinux on p SeriesLinux on z SeriesAIXWindows
z/OS
Linux AIXWindows SUN Solaris
Available AdapterIBM HACMPMSCSVeritas Cluster Services
New inV2.3 New in
V3.1
New inV3.1
IBM Software Group | Tivoli software
© 2008 IBM Corporation38 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
SA Application Manager - Graphical Automation Policy Editor
Graphical editing of policies•SA Application Manager
•SA for Multiplatform
Create, visualize, extend policies
Web application •Runs in Integrated Solution Console (ISC) together with SA Operations Console
Integrated policy checker
XML specific details of policies hidden from the end user
New inV3.1
IBM Software Group | Tivoli software
© 2008 IBM Corporation39 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
SA Application Manager Measurement and Reporting
New inV3.1
IBM Software Group | Tivoli software
© 2008 IBM Corporation40 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
PPRC
z/OS SysplexGDPS
SA AppMan
ClusteredApplications
GDPS K-System
Site 1 Site 2
SA Application Manager
SA Application Manager – Integration with GDPSDistributed Disaster Recovery (DDR)
New inV3.1
IBM Software Group | Tivoli software
© 2008 IBM Corporation41 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
SA Application Manager – Integration with GDPS Scenario: Rolling Disaster with Unplanned Site Takeover
Site 1 Site 2
SA App Man GDPS K System
Linux, AIX, Win, …
local clusters per site or stretched clusters
PPRC
SA App Man GDPS K System
z/OSspanned sysplex
Operator
STOP
START
Win, AIX, … Linux
KILL
Takeover Prompt
KILLSTOP
START
A Potential Disaster is Detected
Decision for a Site Takeover Made
Site Takeover CompleteStop Discretionary Workload
at Recovery SiteStart Production Workload at
Recovery SiteNew in
V3.1
IBM Software Group | Tivoli software
© 2008 IBM Corporation42 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
SA Application Manager - CCMDB Integration Provides data integration with Change and Configuration Mgmt Database (CCMDB) Supports Reconciliation of already discovered Configuration Items (CIs)
New inV3.1
IBM Software Group | Tivoli software
© 2008 IBM Corporation43 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
New tool for end-to-end automation value!
Uses client‘s own data:• Operations staff
• Automation staff
• Education and personnel costs
• Number of composite applications
• Optional input like downtime and customization effort saved
Assumptions:• 2 junior operators per shift
• 20% productivity gain for senior operators
• 50% productivity gain for automation team
• 8h/year less downtime
• Based on client observations, best practices and expert knowledge
Downtime saved in h/yearComposite End-to-End applications 10correct & fast startup 3recovery with cross platform dependencies 2cross cluster/platform move 1Faster problem determination 2Total hours 80
Savings per year Persons K€ /yearTotal
K€/yearjunior level operators 6 75 450senior level operators 3 90 270Automation team 5 110 550Application customization and automation savings 0,56 100 56
hours K€ / hourSavings from higher application availability 80 10 800Savings from less education efforts 71
Total 2196
IBM Software Group | Tivoli software
© 2008 IBM Corporation44 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
More Value
Rating Details
High Reporting of startup, recovery and shutdown timesCan be used to identify and eliminate problems which further
increases uptimegeneric reporting to check SLAs etc.
High Population and currency of CCMDB automatic
Priceless Faster detection of rolling disasters by GDPS (SA AppMan DDR)
High Higher availability of composite applications in disasters due to cross-site move initiated by GDPS (SA AppMan DDR)
IBM Software Group | Tivoli software
© 2008 IBM Corporation45 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Agenda
IBM Tivoli System Automation – Product family and basics
IBM Tivoli System Automation for Multiplatforms
GDPS/PPRC Multi Platform Resiliency for System z (xDR)
IBM Tivoli System Automation Application Manager
Business Continuity Process Manager
IBM Software Group | Tivoli software
© 2008 IBM Corporation46 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
AutomationManager
End to end toolsL
abo
r C
ost
Time
StorageExperts
And tools
Application experts and
tools
Databaseexperts and
tools
Securityexperts and
tools
Networkexperts and
tools
Availability Management
Security and Compliance Management.
Change Management
Release Management
Mainframe experts and
tools
Information Mgmt.
UnixExperts
And tools
IT Dilemma: Managing Cost and Responsiveness Across IT Silos
Integration People, Processes, Tools (end to end), Information
Sense Isolate Diagnose Take Action
Evaluate
Many Businesses Struggle to Manage Composite Applications
IBM Software Group | Tivoli software
© 2008 IBM Corporation47 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
IBM Tivoli Service Management Center for System z Completing System z and Completing your Enterprise
Tivoli Business Service Manager• Improve the alignment of IT objectives and business objectives
Tivoli Application Discovery & Dependency Manager• Improve operational visibility
Tivoli Service Request Manager• Improve end user satisfaction and control over incidents
Tivoli Change & Configuration Management Database• Improve business control
Tivoli Business Continuity Process Manager (BCPM)• Improve business resilience
Tivoli Security Management• Improve business integrity
NEWNEW
NEWNEW
NEWNEW
NEWNEW
NEWNEW
NEWNEW
IBM Software Group | Tivoli software
© 2008 IBM Corporation48 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Business Continuity Process Integration
IT Service Continuity
Install, configure RTO and RPO
Identify the Business Requirements
Service LevelManagement
ExposeSLA, OLA
Incident Management
Investigate & Diagnose Incidents
Prioritize Incidents by
business impact
Incident Resolution &
Recovery
Incident Management
Investigate & Diagnose Incidents
Prioritize Incidents by
business impact
Incident Resolution &
Recovery
Create and Maintain Service Continuity Strategy
Create and Maintain Service Continuity Plan
Prepare IT Service Continuity capabilities
Execute IT Service Plan
Outage Analysis
Plan Backup Capacity
CapacityManagement
Change for IT Redundancy
ChangeManagement
Identify the IT processes for
the critical business
(automated) Recovery
ChangeManagement
Estimate risk and vulnerability
Identify technical options (RTO and RPO per platform
Estimate the impact of a loss or degradation
Approval to declare a Disaster
Identify products, and services for
RTO and RPO configurations
Understand Define Plan Install/Configure/Test Monitor Execute
Verify RTO and RPO
Evaluate e2e RTO/RPO
Understand dependencies
RTO and RPO simulation and verification
Automated Recovery
Declare RecoveryCompletion
Declare a Disaster
IBM Software Group | Tivoli software
© 2008 IBM Corporation49 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Peo
ple
Workflow
Pla
nContext
IBM Tivoli BCPM
Infrastructure
CI
StartOutageAnalysis
Approval Appr.? Stop
DeclareDisaster
Verify Recovery
ExecutePlan
DeclareDisasterFinished
Stop
Auto-Approve
?
Auto-Approve
StartOutageAnalysis
Approval Appr.? Stop
DeclareDisaster
Verify Recovery
ExecutePlan
DeclareDisasterFinished
Stop
Auto-Approve
?
Auto-Approve
Disaster Recovery Workflow
Analyst
Approver
Manager
Planer
Recovery PlanHistory
adapts...
IBM Software Group | Tivoli software
© 2008 IBM Corporation50 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
DR testing Challenges Test Coverage
• Processes (IT and non IT)
• IT Redundancy and Recovery Setup
– Storage/Data, CFs, CDS, Servers, Network, Databases, Application– Sites
• IT Management System
– (GDPS, HACMP etc. Monitoring, Automation/Scripts) Challenges
• How can we test/simulate disasters (significant outages)
– to get a reasonable test coverage– and minimize the impact to the active production
Reasons• Audit the efficiency of the DR plan
• Benchmark BC recovery
• Rehearsal of BC recovery
• Document compliance to regulations
• ….
How can I test IT recovery without executing it ??
IBM Software Group | Tivoli software
© 2008 IBM Corporation51 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
DR Test Cycle Test Planning
• Define Outage Scenario– What is broken and why is it broken (e.g. Site because of fire)
• Define Target/Objectives– Test target, (Process (People) , IT infrastructure, IT Management System) – How do measure success (e.g. time, data quality)
• Define Test Type and Rules– Paper only, Human part only, IT test transactions, FlashCopy restarts – Adhere strictly to planned procedure– Document required deviations
• Define People/Role (Administration, Observation, Documentation)• Define Schedule (Date/Time/Duration)
Test Execution• Decide/Approve/Declare the Test Start/End• Step thru
Test Results• Document results• Plan the changes• Feed experience for next test cycle
IBM Software Group | Tivoli software
© 2008 IBM Corporation52 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
GEO112E
NetView GDPS
Context CI
IT Infrastructure
Test CI Data
CI
History
SAIOM
Simulation / Test History
TestPlan
RecoveryPlan
Workflow
ApproveproposedRecovery
Notifyimpactedpeople
VerifyRecovery
CompleteRecovery
AnalyzeOutage
Collaboration
Crisis management - based on CCMDB / BCPM
IBM Software Group | Tivoli software
© 2008 IBM Corporation53 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
SA Integrated Operations Management (SA IOM )
Automatic Alert forwarding to the person(s) „in charge“
• Flood, fire, abended application Comprehensive set of notification
methods• SMS, voice, pager, email, script
API Interface Built-in escalation if person does
not acknowledge in time Web-based UI to
• define escalation policy
• Acknowledge alerts
• Watch active alerts
IBM Software Group | Tivoli software
© 2008 IBM Corporation54 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Sample Customer Roadmaps towards BC
1) SA z/OS2) SA MP, HACMP3) SA AppMan4) custom DR scripts5) CCMDB6) BCPM
SA z/OS
CCMDBBC PM …
e2e data
mgmt
SA AM – GDPSIntegr.
xDR
TSMTPC-R SA AppMan
GDPS
VCS, MSCS
HACMP
SA MP
OPS/MVS
…AvailPM
…
cust scripts
…
SA z/OS
CCMDBBC PM …
e2e data
mgmt
SA AM – GDPSIntegr.
xDR
TSMTPC-R SA AppMan
GDPS
VCS, MSCS
HACMP
SA MP
OPS/MVS
…AvailPM
…
…
SA z/OS
CCMDBBC PM …
e2e data
mgmt
SA AM – GDPSIntegr.
xDR
TSMTPC-R SA AppMan
GDPS
VCS, MSCS
HACMP
SA MP
OPS/MVS
…AvailPM
…
cust scripts
…
1) OPS / MVS2) GDPS with SA z/OS3) xDR with SA MP
1) SA z/OS2) HACMP, VCS, MSCS3) GDPS4) xDR with SA MP5) SA AppMan6) SA AM – GDPS integration7) CCMDB8) BCPM
custscripts
Examples for incremental growth of HA/DR functionality
IBM Software Group | Tivoli software
© 2008 IBM Corporation55 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
End of Presentation
Thank you very much for your attention
http://www-306.ibm.com/software/tivoli/partners/index.jsp
IBM Software Group | Tivoli software
© 2008 IBM Corporation56 October 2008Business Continuity Advances - IBM Tivoli System Automation Family
Links• Home pages
– SA z/OS http://www.ibm.com/software/tivoli/products/system-automation-390/
http://www-03.ibm.com/servers/eserver/zseries/software/sa/– SA MP http://www-306.ibm.com/software/tivoli/products/sys-auto-multi/– SA AppMan http://www-306.ibm.com/software/tivoli/products/sys-auto-app-mgr/– BCPM http://www-306.ibm.com/software/tivoli/products/business-continuity-process-mgr
• SA MP Data sheet: http://www.ibm.com/common/ssi/fcgi-bin/ssialias?infotype=pm&subtype=sp&appname=SWGE_TI_PA_USEN&htmlfid=TID10427USEN&attachment=TID10427USEN.PDF
• SA MP Interactive Flash Demo: http://www-306.ibm.com/software/tivoli/library/demos/sa-linux.html
• SA MP documentation: http://publib.boulder.ibm.com/tividd/td/IBMTivoliSystemAutomationforMultiplatforms3.1.html
• All SA MP STE webcasts: http://www-950.ibm.com/search/SupportSearchWeb/SupportSearch?action=search&pageCode=SBSXP&brand=tivoli&sortBy=3&pageNumber=1&searchTerms=tste_webcast&products=&addSearchTerms=SAMP&submit.x=8&submit.y=10
• Download SA MP Plug’n Play Automation Modules: http://catalog.lotus.com/wps/portal/tsa
• SA MP Forum: http://www.ibm.com/developerworks/forums/forum.jspa?forumID=1175
• Data sheet: http://www.ibm.com/common/ssi/fcgi-bin/ssialias?infotype=pm&subtype=sp&appname=SWGE_TI_PA_USEN&htmlfid=TID14019USEN&attachment=TID14019USEN.PDF
• Redbook „End-to-end Automation with IBM Tivoli System Automation for Multiplatforms” http://www.redbooks.ibm.com/abstracts/sg247117.html?Open
• SA AppMan documentation: http://publib.boulder.ibm.com/tividd/td/IBMTivoliSystemAutomationforApplicationManager3.1.html
• Webcast: “High Availability and end-to-end automation of AIX, Windows, Linux and z/OS applications using Tivoli SA”: https://cpc.on.intercall.com/confmgr/view_stored_doc.jsp?docId=9179101159285987505108513435&docType=recording
• SA AppMan self-running demo: http://www-01.ibm.com/software/tivoli/library/demos/tivoli-system-automation.html
• SA AppMan forum: http://www.ibm.com/developerworks/forums/forum.jspa?forumID=1304