Fault Tolerant CORBA (FT-CORBA) - Modeling and Analysis
Post on 25-Feb-2016
79 Views
Preview:
DESCRIPTION
Transcript
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 1
Fault Tolerant CORBA(FT-CORBA) -
Modeling and AnalysisIstván Majzik
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems
June 2000
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 2
Introduction• Basis:
– FT-CORBA specification– UML-based automatic dependability modeling
• Topics:– Support to construct optimal FT-CORBA schemes– Evaluate existing architectures
• Part I: The FT-CORBA proposal• Part II: UML-based dependability analysis• Part III: Dep. modeling of FT-CORBA
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 3
Part I
The FT-CORBA Proposal
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 4
CORBA• OMG CORBA: standard of open OO systems
– Provides transparent access to services of remote objects (like local method calls)
– ORB: Object Request Brokercommunication of requests/responses(location, activation, parameter passing etc.)
• IOR: interoperable object reference• GIOP: general inter-ORB protocol• IIOP: Internet inter-ORB protocol
– IDL: Interface definition languageconsistency between client and server interfaces
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 5
FT-CORBA
• Goal: Fault tolerance in CORBA environment
• History:– April 1998: Request for Proposal issued– October 1998: Initial submissions– December 1999: Joint revised submission
by Ericsson, Inprise, Iona, Lucent, Oracle, Sun,...
– April 2000: Final adopted specification
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 6
FT-CORBA Concepts• Avoiding SPOF of single (server) objects• Fault tolerance by entity redundancy,
fault detection and recovery– creation of (server) object groups– infrastructure to maintain object replicas
• Basic properties:– replication transparency
(access independent of number/location)– failure transparency
(access independent of faulty server objects)
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 7
Fault Tolerance Domains• FT domain:
– Object groups of server object replicas– Single Replication Manager
• Object groups:– different hosts– single object per host
• Replication Manager:– Creation and management of object groups– Support of application-controlled management
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 8
Fault Tolerance Domain
Domains, object groups, hosts and replicas
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 9
Architecture Overview
• Set of CORBA objects to support FT– Replication Manager– Fault Detector– Fault Notifier– Fault Analyzer
• ORB extensions– logging mechanism– recovery mechanism
• Commercial implementations?
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 10
Fault Tolerance Infrastructure
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 11
Replication Management• Infrastructure controlled case:
– application: create_object() method of the RM– RM: invokes local factory objects on hosts– RM manages membership, consistency
• Application controlled case:– application’s responsibility to manage replicas
• Parameters:• ReplicationStyle: stateless, cold / warm passive, active• MembershipStyle• ConsistencyStyle• InitialNumberReplicas, MinimumNumberReplicas
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 12
Fault Detection and Notification
• Fault model: – object crash (incorrect results are not tolerated)
• Fault detection by polling– application objects inherit the PullMonitorable
interface: is_alive() method– Fault Detector invokes it periodically– hierarchy of fault detectors
• Fault notification and fault analysis• Parameters:
– FaultMonitoring (Style, Granularity, IntervalAndTimeout)
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 13
Logging and Recovery• Application objects inherit:
– Checkpointable interface: get_state(), set_state()– Updateable interface: get_update(), set_update()
• Logging Mechanism:– storing GIOP messages– periodically storing state of the objects
• Recovery Mechanism:– restore object state and retrieve stored messages
• Parameters:– CheckpointInterval
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 14
Client Failover
• Identification of object groups:– IOGR: interoperable object group reference– multiple IIOP profiles
addressing object group members or gateways
• Basic mechanisms of the client ORB:– retry all alternative IIOP profiles– transparent reinvocation of requests
(“at most once” execution semantics at the server)– heartbeating of the server
IIOP IIOP IIOP
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 15
Part II
Dependability Modeling of Object-Oriented Systems
Described in UML
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 16
Dependability AnalysisApproach by A. Bondavalli, I. Majzik, I. Mura
HIDE - High-level Integrated Design Environment for DependabilityESPRIT Open LTR No. 27493
• From UML-based models (class, object, deployment diagrams)
to Timed Petri Netsstandard PN evaluation tools can be used
• Supports– comparison of design choices– identification of bottlenecks
• System-wide, structural model
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 17
Modeling Approach
1. UML model: Diagrams with extensionsstereotypes to identify roles (variant, tester, ...)tagged values to assign parameters
2. Intermediate model: Simplified structureelements: software, hardware, with/wo statesdependencies: „uses the service of” „is
composed of”class based redundancy fault tree
3. Dependability model: Timed Petri netsub-nets for elements and dependencies
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 18
Failure/Propagation Sub-models
H E
fault
F
latency
p
1-p
prop
no_prop
restart
E1
E2
H2
<<SF-SW>>
UML model elements Petri net modules
O1
O2 O1
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 19
Repair Sub-model
E
latency
m(E)=1H F
fault
P1
P2
P3
t0t1
perm
transimplicit
explicit m(P1)=0m(P1)=1
<<SF-HW>>
UML model Petri net module
O1
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 20
Redundancy Sub-models
Variant1failure
Variant2failure
RedundancyManager
failure
Subsystem failure
G
P
SF
P
G
RM
V1 V2
UML model Fault tree Petri-net
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 21
Part III
Dependability Modeling of FT-CORBA Architectures
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 22
Approach• UML models:
– identification of elements/structures– additional parameters support of automatic modeling
• Tailoring to FT-CORBA– subnets to specific mechanisms– based on the parameters
• Restrictions: – non-replicated client, static structure– infrastructure controlled replication management
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 23
UML Modeling• Identification of elements/structures
– Fault Tolerance Domain: package• independent of deployment
– Object groups: sub-package– Roles: stereotypes
• FT-CORBA properties as tagged values– ReplicationStyle– MembershipStyle– ConsistencyStyle– FaultMonitoring (Style, Granularity, Interval)– (Initial, Minimum) NumberReplicas
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 24
Overall Structure
FT Domain Alpha
Domain2
FTI
RM FN FD
OG4OG3OG2OG1
S11 S12 FD1
Domain1<< >>
<< >>
<< >>
C1 C2
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 25
Modularity• Available building blocks:
– failure subnet– propagation subnet– repair subnet– fault tree
• Sub-models in FT-CORBA:1. Client failover2. Server object failure3. Fault management (detection and notification)3. Recovery (replication management)
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 26
1. Client Failover
• Semantics:– Primary is tried first– Failover conditions: „crash”
• Communication failure• No response
No failover: erroneous response– No failure exception until all profiles have been tried
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 27
Dependability Sub-model
Fault tree (passive replication):– Top event: Client failure– Basic events:
• Server object crash• Server object erroneous response
– Composite events (OR): number n of profiles • S1 (primary) erroneous• S1 crash AND S2 erroneous• S1 crash AND S2 crash AND S3 erroneous• ... • S1 crash AND S2 crash AND ... AND Sn crash
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 28
2. Server Object Failure• Distinction of failures:
– Crash Failover in client Error detected in the object group
– Erroneous response (commission fault) Propagated to clients,
application-specific error detection
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 29
Dependability Sub-model
• Failure process:– failure subnet– distinguished cases: crash/erroneous response
• Propagation subnets– standard subnets (toward the client fault tree)
H E
fault rateF
latency
C ER
p 1-p
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 30
3. Fault Management
• Fault detection+notification: Chain of events– Source: Fault Detector
• latency = MonitoringInterval• coverage depends on MonitoringGranularity:
– each member / single per host / single per host and type
– Propagation: Fault Notifier(s)• communication failures
– Destination: Replication Manager• Hierarchy of Fault Detectors• Infrastructure objects: Replication is possible
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 31
Dependability Sub-model
• Error detection delay– timed PN transition
• Fault notification subsystem– fault tree (AND)
• Replicated infrastructure objects– local fault trees (AND)
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 32
4. Recovery in the Object Group• Triggered by the Fault Notifier
in the Replication Manager• Goal: Maintain the number of replicas
– crashed object is removed– creation of new replica, restoring state– only a single replica on a given host!
• Repair is possible if:– current host is fault-free– current host is faulty, but there are available hosts
i.e. number of hosts >= NumberReplicas
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 33
Dependability Sub-model
• Repair subnet: Explicit repair– latency: CheckpointInterval, ReplicationStyle
• Recovery of the replica:– Static deployment:
Standard repair subnet– Pool of identical hosts: Logic condition for repair
Free hosts (PN place)• marking increased by host repair and server object crash• marking decreased by host crash and server object repair
Guard on the transition for explicit repair
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 34
Overall Structure of Subnets
Notification
Prop.
Client Fault Tree
S1 err
Prop.
S1 crash
Recovery Repair
NumberReplica
FaultMonitoringGranularity FaultMonitoringInterval
ReplicationStyle CheckpointInterval
Prop. Prop.
Prop. Prop.
Budapest University of Technology and EconomicsDepartment of Measurement and Information Systems 35
System-wide Dependability Model• Analysis of the Petri-net:
– standard tools (SPNP, PANDA, ...)
• Sensitivity analysis– system-wide reliability, availability
Optimal selection of FT-CORBA parameters– replication (membership, consistency) styles– number of replicas– monitoring granularity, interval
top related