1 Stracener_EMIS 7305/5305_Spr08_04.17.08 System Availability Modeling Dr. Jerrell T. Stracener, SAE Fellow Leadership in Engineering EMIS 7305/5305 Systems Reliability, Supportability and Availability Analysis Systems Engineering Program Department of Engineering Management, Information and Systems
36
Embed
Stracener_EMIS 7305/5305_Spr08_04.17.08 1 System Availability Modeling Dr. Jerrell T. Stracener, SAE Fellow Leadership in Engineering EMIS 7305/5305 Systems.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Stracener_EMIS 7305/5305_Spr08_04.17.08
System Availability Modeling
Dr. Jerrell T. Stracener, SAE Fellow
Leadership in Engineering
EMIS 7305/5305Systems Reliability, Supportability and Availability Analysis
Systems Engineering ProgramDepartment of Engineering Management, Information and Systems
2
Stracener_EMIS 7305/5305_Spr08_04.17.08
Availability Modeling
• Requirements and Figures of Merit• Analytical versus Simulation Modeling• Availability Model Development• Blue Flame Aircraft Case Study• Summary and Discussion
3
Stracener_EMIS 7305/5305_Spr08_04.17.08
Requirements and Figures of Merit
Why Model?
4
Stracener_EMIS 7305/5305_Spr08_04.17.08
Availability Analysis
provides a mathematical basis for evaluating system design and development decisions based on system level performance measures in order to influence the air vehicle design concurrently with support system design.
5
Stracener_EMIS 7305/5305_Spr08_04.17.08
System Design Evaluation Categories
Operational Effectiveness Evaluation
“To what degree does this system satisfactorily support mission accomplishment when used by representative personnel in the expected or potential environment for operational employment of the system considering organization, doctrine, tactics, survivability, vulnerability, and threat?”
Operational Suitability Evaluation
“To what degree can this system satisfactorily be deployed considering availability, compatibility, transportability, interoperability, reliability, wartime usage rates, maintainability, safety, human factors, manpower supportability, documentation and training requirements?”
Functional Effectiveness Evaluation
“How and to what degree will this system satisfactorily contribute to the required mission(s) in the predicted operational environment?”
• Analytical Representations– Mathematical formulas and symbolic models– May use computers to process the formulas
• Computer Simulations– Imitation of the physical phenomena(movement,
war, performance overtime) using computer generated activities and results
– human decision making represented by pre-programmed and/or probabilistic decision rules
• Assemblage of Gaming People and Tools– Human-based “game playing” to achieve insights
(e.g. war games)
• Field Experiments– Replications of a physical situation under controlled
and limited scale environments to estimate total system level performance
10
Stracener_EMIS 7305/5305_Spr08_04.17.08
When simulation models make sense
• When mathematical models do not exist, or analytical methods of solving them have not yet been developed
• When analytical methods are available, but mathematical solution methods are too complex to use
• When analytical solutions exist and are possible, but are beyond the mathematical capabilities of available personnel
11
Stracener_EMIS 7305/5305_Spr08_04.17.08
When simulation models make sense
• When it is desired to observed a simulated history of the process over a period of time in addition to estimating relevant parameters
• When it may be the only possibility because of difficulty in conducting experiments and observing phenomena in their actual environment
• When time compression may be required for systems over long time frames
12
Stracener_EMIS 7305/5305_Spr08_04.17.08
Advantages of Simulation
• Permits controlled experimentation with:– consideration of many factors– manipulation of many individual units– ability to consider alternative polices– little or no disturbance of the actual system
• Effective training tool• Provides operational insight• May dispel operational myths
13
Stracener_EMIS 7305/5305_Spr08_04.17.08
Advantages of Simulation
• May make middle management more effective
• May be the only way to solve problem
14
Stracener_EMIS 7305/5305_Spr08_04.17.08
Disadvantages of Simulation
• Costly (very costly?)• Uses scarce and expensive resources• Requires fast, high capacity computers
(use of PC’s?)• Takes a long time to develop• May hide critical assumptions• May require expensive field studies• Very much dependent on availability of
Theater Simulation of Air Base Resources (TSAR)-Rand Corp.
Douglas Aircraft Company Availability Model (DACAM) System Inventory Analysis Model (SIAM)
– More detailed A/R/S ModelsModified Logistics Composite Model (LCOM)-USAFComprehensive A/C Support Effectivenes Eval.
(CASEE)Model -USNavy
16
Stracener_EMIS 7305/5305_Spr08_04.17.08
When Simulation Models Make Sense(An Analyst’s Checklist)
• When mathematical models do not exist, or analytical methods of solving them have not yet been developed
• When analytical methods are available, but mathematical solution methods are too complex to use
• When analytical solutions exist and are possible, but are beyond the mathematical capabilities of available personnel
• When it is desired to observe a simulated history of the process over a period of time in addition to estimating relevant parameters
• When it may be the only possibility because of difficulty in conducting experiments and observing phenomena in their actual environment
• When time compression may be required for systems or processes over long time frames
17
Stracener_EMIS 7305/5305_Spr08_04.17.08
System Life Cycle Utility of Models/AnalysesConcept Explore.
DEM/VAL FSED PRODUCT DEPLOY
Statistical Analysis Capability Low Medium High High HighReliability Growth Models Low Medium High - -
LCC/CERs/O&S/DTC Models Low* Medium* High Medium LowFMECA Process Low* Medium* High Low** Low**
Basic Rel & Maint Anal. Tools Medium* Medium* High Low -Airbase Operations Model Low* Low* High Low** Low**LCOM/Airbase Ops Model Low* Low* High Low** Low**
Integ. CALS Status Reporting Low* Low* High High LowMsn Effect. & Supportability Medium* Medium* High Low -
Probability of Mission Success Medium* Medium* High Low -Analyt. Avail. (Markov) Models Medium* Medium* High - -Paramet. R&M Prediction Mod Medium* High* High - -
Enhanced ILS/LSAR System Low* Medium* High Medium Low**Reliability Centered Mainten. - Low* High Medium Low**
Enhanced RCM Process - Low* High Medium Low**High Level A/R/S Models High* High* Medium Low** Low**
LCOM w/Model Mgmt. System Low* Low* High High** High**Comprehensive Autom. Supp. Low* Medium* High Medium Low**
RMS Design Training Pkgs. Low Medium High Medium LowCALS/CASE (Full Up Capab.) Low* Medium* High High Medium*
* - High level analysis ** - ECP/Changes/Problem Resolution
18
Stracener_EMIS 7305/5305_Spr08_04.17.08
Thoughts To Remember
• The overall objective of availability modeling and analysis is to provide support to the system design, development, and deployment process in order to influence system design by considering all aspects of its reliability, maintainability, and support system characteristics
• The objective remains unaffected by the choice of using one model solution technique (e.g. simulation) over the other.
• The efficacy of choosing one method over the other will be influenced primarily by outside factors (e.g. cost, schedule, availability of data, personnel and facility capabilities).
19
Stracener_EMIS 7305/5305_Spr08_04.17.08
Availability Model Development
20
Stracener_EMIS 7305/5305_Spr08_04.17.08
Model Development Overview
• Analysis Objectives• Analysis Planning• Development Approach• Development Considerations• Inputs and Outputs• Data Requirements• Algorithm Development• Implementation Examples
21
Stracener_EMIS 7305/5305_Spr08_04.17.08
RMS Analysis Objectives
• Specification Requirements Evaluation– Requirement Integration - Conflicts? Attainable?– Verify and Demonstrate Compliance– Verify Demonstrate Adequacy of Logistics Support
• Support System Design Influence– Evaluate Impacts of Changes to Operation and Maintenance Concepts– Analyze & Evaluate Operational Suitability– Support Functional Trade-off Analyses on Alternative Designs
• System Design Assessment– Examine the Total Picture at the System Level– Address Impacts of All Variables at once– Evaluate Impacts of Flight/Scenario/Usage Rate Changes
• Management Visibility– Provide Useful Predictions for All Levels of Management– Assist Management in Identification and Resolution of Reliability,
Maintainability, and Supportability Issues
22
Stracener_EMIS 7305/5305_Spr08_04.17.08
RMS Analysis Objectives by Program Phase
• Concept Definition– Support Contractual Requirements Analysis
• Examine Operations, Maintenance, and Support Concepts– Support Design Concept Trade-off Studies– Identify Cost, Schedule, Risk, and Support Drivers
• Demonstration/Validation– Refine Concept Definitions– Support Requirements Allocation Process– Provide Capability to Influence Design– Estimate Fielded System performance Levels
• Full-Scale Engineering Development– Support Detailed Trade-off Studies– Establish Support System Requirements Baseline– Assess/Validate Operations, Maintenance, Support Concepts
• Production and Deployment– Asses Fielded System Performance Levels– Refine Support Concepts/Levels– Identify System Improvement Requirements
23
Stracener_EMIS 7305/5305_Spr08_04.17.08
RMS Analysis Planning Considerations
• Critical Issues
•Objectives
•MOEs & MOSs
•Success Criteria
•Schedule
• Test Design
•Analysis Plan
•Data Collection & Management Plan
•Test Execution Plan
•Documentation Plan
•Test and Evaluation Master Plan
•Where does data come from?
–Experiment?
–Field tests?
–Previous experience?
–Simulation?
–Other resources?
•What will data be used for?
•How will data be collected and managed?
•What tests/simulations need to be executed, and when?
•How will results be dev. and rec?
•How does everything fit together to meet the system test & eval. objectives?
Evaluate A/R/S Analysis Reqs.
Develop Test /Analysis Plans
24
Stracener_EMIS 7305/5305_Spr08_04.17.08
RMS Model Development Approach
1. Define Model Elements and Specificationsi. Operational Activity Elemanet Specificationsii. System State Conditions and Attribute Specificationsiii. Operational Activity Demand Generationiv. System Component Level of Detail Determinationv. Support System Resource Definition and Specifications
2. Define Model Structurei. Model Processing Definition(s)ii. System Failure Processingiii. System Unscheduled Maintenance Processingiv. Model Inputsv. Model Outputs
3. Implement Model Structure on the Computeri. Model Activitiesii. Model Output Measure Calculation Implementation
4. Perform Full Model Test & Eval. Using Sample Data5. Install Model at User Site and Perform Checkout,
Train Users
25
Stracener_EMIS 7305/5305_Spr08_04.17.08
Probabilistic Modeling(probabilistic analysis)
• Purpose: To simulate probabilistic situations using a random number generator and the cumulative probability distribution of interest.
• Example: Distribution of unscheduled maintenance times:no action required (none), repair in place (RIP), remove and replace (R&R), and cannot duplicate (CND)
• Data Input/Output Formats• Data and Output Result Configuration Management &
Control• Input/Output Data Approval by Management• Baseline and Excursion Data Definitions/Conditions• Data Screening/Editing Capabilities• Model Restart Capabilities• Ease of Development and Modification• Transparency to the Users (changes to system and
data)• Degree of integration with other models and Analyses• Convenient Man-in-the-Loop Interfaces• Growth/Flexibility/Change Capabilities• Others
27
Stracener_EMIS 7305/5305_Spr08_04.17.08
Typical RMS Model Requirements
• Work Unit Code (WUC) Structure– Total system WUC structure– Two Digit level definitions (or to levels of interest)
• Probability Distributions fro Activity Times (by WUC)– Mission durations and types– Trouble-shooting times– On/off aircraft repair times– Remove, replace, checkout times– Delay times (spares, personnel, equipment)– Service and turnaround times– Preflight and return service times
• Probabilities (by WUC)– Probability of in-flight failure (gripe)– Spares, personnel, equipment availability – when called– On equipment vs. off-equipment repair rates– No defect found rates
28
Stracener_EMIS 7305/5305_Spr08_04.17.08
Input Data Sources & Parameters
• AFR 66-1 (Maintenance Data Collection System- MDCS) Data Elements
• CORE Automated Maintenance System (CAMS)
• AFR 65-110 (air Vehicle Inventory Status and Reporting System (AVISURS)
• Others
• “Reliability” in terms of MTBM– Types 1,2, & 6
• “On-equipment” & “Off-equipment” Maintenance Action Definitions:– Repair in Place– Cannot Duplicate– Bench Check –- Repair– Bench Check –- Serviceable– Not Repairable in this Station
• Work Unit Code Definitions• Others
Air Force RAM Data Sources Model Data Element Definitions derived from Air Force Terms
• Availability Parameters– Average mission capable rates (full, partial, not capable)– Instantaneous mission capability status at any time in the
simulation/analysis period
• System Level Performance Parameters– Average downtime per sortie– Average unscheduled maintenance time– Percent of scheduled sorties accomplished (over time)– Number of sorties cancelled due to pre-sortie failure– Number of unscheduled maintenance actions required
• Maintenance Resource Utilization Statistics– Total resource hours used during simulated period (by resource type)– Maximum number in use at any time during simulation– Total number of subsystem spare parts used
30
Stracener_EMIS 7305/5305_Spr08_04.17.08
Characteristics of PC-based Modeling
• Can provide stochastic network processing with discrete events using simulation languages implemented on PC’s (SLAM II)
• Can simulate system operational environments:– Basic operations and maintenance processing defined by
established input networks– Specific task information (times, required resources, task
attributes, etc.) supplied through input data• Will treat system maintenance simulated at line replaceable
unit (LRU) level of detail with input and output data aggregated at the subsystem level of detail
• Provides real-time system capability assessment over a wide range of design and development parameters with relatively small set of input data required
• Use of real-time graphics capabilities promotes model understanding and display of results of different execution conditions and constraints
• Portability permits use in remote and dispersed locations for examining impacts of local environmental and support conditions
31
Stracener_EMIS 7305/5305_Spr08_04.17.08
What Talents are Required?
• System Operators– Develop operational and support requirements and
concepts– Develop measures of effectiveness (MOEs) and
supportability (MOSs)
• System Modelers– Develop system-specific modeling and analysis
Early Oper. Suitability Anal. Support Req. AssessmentField/Deployed Avail. Anal.
Availability-Oriented Provisioning Model (AOP)
Tracking/Data Relay Satellite Station (TDRSS);Class. Sys.
Spare Part Prov. Req.Optimum Spare Part Prov. For Availability
Logistics/Maint. Attack Model(LOGATAK,MACATAK)
Defense Nuclear Agency; US Army Logistics Center
Effects of Enemy Interdiction on Logistics Support Systems
Network Repair Level Analysis (NRLA) Model
Tactical Remote Sensor System (TRSS) Optimum Repair Level Analyses
Previous Availability Model Applications
33
Stracener_EMIS 7305/5305_Spr08_04.17.08
Summary and Conclusion
34
Stracener_EMIS 7305/5305_Spr08_04.17.08
The Benefits of Availability Modeling & Analysis
• Availability Modeling and Analysis provides the “glue” which ties system RMS performance evaluation together:– Considers operational environments/stresses– Identifies dominant failure modes’– Balances overall support system performance
• It provides one of the few methods capable of estimating fielded system performance levels during the design and development process.
• Applies to Commercial as well as DoD systems
35
Stracener_EMIS 7305/5305_Spr08_04.17.08
Availability Analysis: A value-added process
• Availability analysis provides the “glue” which ties system RMS performance evaluation together:– Considers operational environments and
stresses– Identifies dominant failure modes– Incorporates repair and replace times
estimates– Evaluates overall support system
performance
36
Stracener_EMIS 7305/5305_Spr08_04.17.08
Availability Analysis: A value-added process
• It provides a rational structure for evaluating system design and development decisions based on system level performance measures.