I-RaCM: A Fully Integrated Risk and Life Cycle Cost Model · framework, providing for a seamless integrated analysis solution. Both newly developed and existing industry-standard

American Institute of Aeronautics and Astronautics

1

I-RaCM: A Fully Integrated Risk and Life Cycle Cost Model

Dominic DePasquale1 and A.C. Charania2

SpaceWorks Engineering, Inc. (SEI), Atlanta, GA, 30338

SpaceWorks Engineering, a business division of SpaceWorks Engineering, Inc. (SEI),

conceived of the Integrated Risk and Cost Model (I-RaCM) to meet the need for

synchronous cost and risk assessment early in the design of a new system. I-RaCM

encompasses a diverse set of cost and risk tools implemented in the ModelCenter®

framework, providing for a seamless integrated analysis solution. Both newly developed and

existing industry-standard software tools are combined within I-RaCM allowing for rapid

evaluation of life cycle cost, operations, reliability, technology development costs, and

commercial business case viability. Tools currently integrated within I-RaCM include the

NASA Air Force Cost Model (NAFCOM), Galorath SEER-H, SpaceWorks Engineering’s

Facilities and GSE Operations Assessment (FGOA) Tool, and a reliability calculation tool

using Fault Trees and Event Sequence Diagrams. Also included is an initial version of a new

Technology Cost Estimation (TCE) tool developed to estimate technology investment and

development costs to achieve TRL-6 status based on a simple set of relevant inputs. Finally,

the I-RaCM platform contains a custom created system level cost/risk aggregation tool,

called Stack’em, which collects, post-processes, and graphically summarizes key outputs.

Several novel cost visualization techniques have been conceptualized for Stack’em, providing

designers with comprehensive insight into major cost and risk outputs, along with a budget

optimization process that automatically adjusts program expenditures and schedule to fit

under a fixed budget curve.

This paper details the current functionality and toolset of I-RaCM. Results from four

case studies demonstrating the capabilities of I-RaCM through analysis of a notional

present-day NASA lunar exploration architecture are also presented. The entire integrated

environment of I-RaCM is capable of linking with performance disciplinary tools through

pre-established input and feedback interfaces. SpaceWorks Engineering plans to continue

expanding I-RaCM with the addition of more advanced reliability analysis, availability and

performability tools, and discrete event simulation of operations processing.

Nomenclature

ABM = Agent Based Modeling LCC = Life Cycle Cost

CABAM = Cost and Business Analysis Module LEO = Low Earth Orbit

CER = Cost Estimating Relationship LLO = Low Lunar Orbit

CEV = Crew Exploration Vehicle LOC = Loss of Crew

CaLV = Cargo Launch Vehicle LOM = Loss of Mission

CLV = Crew Launch Vehicle LOR = Lunar Orbit Rendezvous

CM = Command Module LSAM = Lunar Surface Access Module

DD = Design & Development MS = Microsoft

DDT&E = Design, Development, Testing & Evaluation NAFCOM = NASA Air Force Cost Model

EDS = Earth Departure Stage NASA = National Aeronautics and Space Administration

ESAS = Exploration Systems Architecture Study NESC = Nodal Economic Space Commerce [Tool]

ESD = Event Sequence Diagram SM = Service Module

EVA = Extra-Vehicular Activity T&E = Testing and Evaluation

1 Director, Engineering Economics Group, SpaceWorks Engineering, 1200 Ashwood Parkway - Ste. 506, AIAA Member.

2 President, SpaceWorks Commercial, 1200 Ashwood Parkway - Ste. 506, AIAA Senior Member.


2

FGOA = Facilities and GSE Assessment [Tool] TEI = Trans-Earth Injection

FTA = Fault Tree Analysis TIM = Technology Impact Matrix

GSE = Ground Support Equipment TFU = Theoretical First Unit [Cost]

I-RaCM = Integrated Risk and Cost Model TLI = Trans-Lunar Injection

ISS = International Space Station TPS = Thermal Protection System

IVHM = Integrated Vehicle Health Monitoring TRL = Technology Readiness Level

I. Introduction

IABILITY of any aerospace program in the present era of scarce resources and competing national objectives

is ultimately determined by its life-cycle economics. Successful analysis and design of a new system must

therefore balance cost, risk, and performance to develop a product that satisfies customer objectives and is robust to

uncertainty. It is advantageous to fully assess these interrelated metrics concurrently during the conceptual design

phase when the greatest design freedom is available; however performance is typically considered independently of

cost and risk in the traditional early design process. A greater pressure on program managers to deliver more capable

products on schedule and within the available budget has elevated the importance of cost and risk driven design, but

integration into the existing systems engineering process is considerably challenging. The Integrated Risk and Cost

Model (I-RaCM) meets these challenges and delivers the advantages that only a seamless integrated solution of cost

and risk models can provide.

A. Motivation for an Integrated Risk and Cost Tool The ability to understand and accurately forecast the cost and risk associated with major aerospace programs has

an important and expanding role as program managers face increased pressure from commercial, societal, and

political sources. Tightening budgetary constraints and reduced tolerance for cost overruns necessitate accurate

lifecycle cost estimates by program managers and designers. Political pressure and increased public scrutiny also

encourage program managers to communicate realistic estimates of cost and risk associated with a project.

Furthermore, it is often advantageous to make compromises between system performance and cost/risk to best meet

mission objectives. Finally, the movement toward shorter design cycles and the need for immediate decision-making

early in the design process requires that cost and risk insight is quickly and accurately generated and analyzed.

While the need for assessment of cost and risk may be recognized, accurately modeling these complicated

metrics during conceptual design presents several challenges. The addition of cost and risk parameters increases the

size of an already large trade space, and time constraints can prevent full exploration of design options. It is also

difficult to calculate and quantify metrics such as affordability, reliability, and other “ilities” due to a lack of data or

experience. Analyzing sensitive programmatic factors like cost can also expose the design team to additional

scrutiny and concern if the initial cost estimates, prior to optimization, are high relative to the budget.

B. Advantages to an Integrated Risk and Cost Tool

Success with multi-disciplinary performance analysis frameworks has demonstrated the value of integrated tool

sets during conceptual design. SpaceWorks Engineering in particular has previously used integrated performance

models to conceptually design reusable launch vehicles and other applications.1,2,3,4

In addition, SpaceWorks

Engineering has applied multi-disciplinary design approaches to rapidly assess technology investment to various

aerospace systems.5,6,7,8

Cost, reliability, and other “ilities” have been included in these multi-disciplinary

frameworks to a limited extent, providing a foundation in thinking for a fully integrated toolset. I-RaCM offers a

more complete set of “ility” analysis tools, many with higher fidelity and greater flexibility than previously

integrated together. Specific advantages of I-RaCM over current practices include:

1. Complete Cost Picture: I-RaCM provides a more complete picture of cost and risk associated with a project

over its life cycle. Design, Development, Testing, and Evaluation (DDT&E) and Theoretical First Unit (TFU) costs

are common to cost estimates, but often the fixed and recurring costs of operations are neglected. The cost of

failures and the cost of risk reduction are even less frequently accounted for in early cost estimates. Through its

contained toolset, I-RaCM accounts for all major conceivable life cycle costs, and includes risk in the analysis loop.

2. Utilization of Existing Best-of-Class Tools: The integration of multiple tools has several advantages as

opposed to the creation of a single do-all cost and risk application. The use of an integrated tool set results in more

meaningful analysis by allowing for individual best-in-class tools to perform the particular function for which they

are most suited. The availability of readily integrated individual tools also allows for the user to select a specific

V


3

subset of I-RaCM tools to best analyze a particular problem. Upgrades to the integrated model are also easily

accomplished by upgrading the individual tools, and the integrated model maintains its integrity as long as the input

and output links between individual tools remain the same. By comparison, compromises in capability that one

makes to achieve a single do-all application may result in diluting the effectiveness and flexibility of the tool.

3. Interface with Other Design Disciplines: Inputs are often fed to cost analysis tools as the last step in a

traditional, performance oriented, design process. Including cost and risk not just as an afterthought, but as an active

discipline in the design process, can greatly improve the cost to benefit ratio of the final design. I-RaCM will have

the ability to easily interface with performance disciplines implemented in Phoenix Integration’s ModelCenter®

integration framework.

4. Analytical Consistency: A serial human-in-the-loop process for cost and risk analysis is prone to errors in

communication and data entry. An integrated cost and risk process, with established and verified data transfer

protocols between inputs, is more likely to maintain analytical consistency and eliminate transcription errors.

5. Efficiency: An automated tool is more efficient than a traditional human-in-the-loop serial design process.

Changes at any level in the design can rapidly propagate through the cost tools without any need for human

intervention. Design trades can then be automatically assessed in a manner that includes system optimization and

uncertainty assessment. The automated nature of the tools allows for the thousands of iterations necessary for these

tasks to be rapidly accomplished.

6. Investigation of Uncertainty: Sources of risk can be more thoroughly understood with the capability to assess

uncertainty in cost, reliability, and schedule. By defining distributions for key input variables, the uncertainty

associated with outputs can be investigated using probabilistic Monte Carlo analysis. I-RaCM has the ability to

conduct probabilistic uncertainty analysis using PiBlue Software or ModelCenter® plug-in drivers.

7. Optimization: The ability to interface with other design disciplines and the collection of design tools within

one framework allows for design optimization. I-RaCM can utilize a suite of industry standard optimization

algorithms to demonstrate optimization with low cost or risk as an objective function.

II. I-RaCM Overview and Tools

I-RaCM encompasses a diverse set of cost and risk tools implemented in the Phoenix Integration ModelCenter®

framework, providing for a seamless integrated solution. Both newly developed and existing industry-standard

software tools are combined within I-RaCM, allowing for rapid evaluation of life cycle cost, operations, reliability,

technology development costs, and business case viability. ModelCenter® provides the means to link multiple

software tools together, control the tools at runtime, and performs data collection and processing. Development of I-

RaCM is ongoing to expand the comprehensiveness of its cost and risk analysis capabilities. SpaceWorks

Engineering plans to add more advanced reliability analysis, availability calculation, and discrete event simulation

of operations processing. The graphic of Figure 1 depicts the current and planned disciplinary analysis tools within

I-RaCM. The integration of tools denoted as partially current and partially future is in progress. The initial ability to

operate these tools in the ModelCenter® environment has been demonstrated, but they are either not yet fully

developed or not yet fully tailored for I-RaCM.

I-RaCM contains several tools and capabilities that are first-of-their-kind accomplishments for an integrated

modeling environment and the conceptual deign community. The integration of the government-sponsored NASA

Air Force Cost Model (NAFCOM) and Galorath’s SEER-H hardware cost model into I-RaCM is of particular

consequence and value. These two tools are commonly used for cost estimating in the aerospace industry, and the

authors are unaware of any previous integrated model containing their full versions. SpaceWorks Software’s

commercially available Remix software tool allows for the full, GUI-executed, versions of NAFCOM and SEER-H

to interface with ModelCenter®. Another unique capability included is an initial version of a new Technology Cost

Estimation (TCE) tool developed to estimate technology investment and development costs to achieve Technology

Readiness Level (TRL) 6 status based on a simple set of relevant inputs. The I-RaCM platform also contains a

custom created system level cost/risk aggregation tool (Stack’em) which collects, post-processes, and graphically

summarizes key outputs. Several novel cost and visualization techniques have been conceptualized for Stack’em,

providing designers with comprehensive insight into the major cost and risk outputs. Stack’em also contains a a


4

budget optimization capability that automatically adjusts program expenditures and schedule to fit under a fixed

budget curve.

In addition to disciplinary analysis tools in I-RaCM, other tools are available to define program life cycle inputs

and conduct probabilistic simulation. The Campaign Definition Spreadsheet allows the user to lay out a year-by-year

mission profile for all elements of the architecture or system. The Technology Impact Matrix (TIM) allocates the

impacts of selected technologies amongst the applicable architecture elements. Together with TCE, the TIM

provides the capability for I-RaCM to assess the development cost of technologies themselves as well as the

development and production cost implications of the element to which the technology is applied. Finally, the

ProbWorks© and OptWorks© driver tools allow for probabilistic analysis of the entire integrated model and global

optimization respectively. Other tools exist to perform probabilistic analysis and optimization within ModelCenter®

and these could be easily substituted if preferred.

The complete set of the tools currently integrated within I-RaCM, as well as those planned for inclusion in the

future, are listed in Table 1. While all of the I-RaCM tools listed here are available, not all must be used when

conducting an analysis. I-RaCM provides the capability to easily link any of these tools, but instances of I-RaCM

may exist with only a subset of the available tools, depending on the problem being analyzed. Detailed descriptions

of each of the current and partially integrated tools listed in Table 1 are given in the following sections.

NAFCOM

SEER-H

Development and Production Cost

NAFCOM

SEER-H

Development and Production Cost

Facilities and Ground Ops Analysis (FGOA)

Facilities and Operations Cost

Facilities and Ground Ops Analysis (FGOA)

Facilities and Operations Cost

Cost And Business Analysis Module (CABAM)

Nodal Economic Space Commerce (NESC) Tool

Business Case Evaluation

Cost And Business Analysis Module (CABAM)

Nodal Economic Space Commerce (NESC) Tool

Business Case Evaluation

Event Sequence Diagrams

Fault Trees

Reliability

Event Sequence Diagrams

Fault Trees

Reliability

Technology Cost Estimator (TCE)

Technology Cost

Technology Cost Estimator (TCE)

Technology Cost

Descartes Discrete Event Simulation in Arena

Operations Scheduling and Simulation

Descartes Discrete Event Simulation in Arena

Operations Scheduling and Simulation

Probabilistic Risk Assessment

Risk / Reliability and Consequences

Probabilistic Risk Assessment

Risk / Reliability and Consequences

Stochastic Petri Nets

Availability and Performability

Stochastic Petri Nets

Availability and Performability

Data Handlingand Integration

Stack’em

Output Visualization and Exploration

Stack’em

Output Visualization and Exploration

I-RaCM: Integrated Risk and Cost Model Legend:

<Preferred Tool>

<Function>

<Preferred Tool>

<Function>

Current

Future

PHX Integration

ModelCenter

Figure 1: I-RaCM Disciplinary Analysis Tools


5

A. PHX ModelCenter® Integration Environment

Tool Name: ModelCenter®

Function: Software Tool Integration

Platform: Windows® Executable

Origination: Developed and commercially sold by Phoenix Integration, Inc.

Description: ModelCenter® allows for the coupling of various disciplinary software tools into one integrated

design model. ModelCenter® provides a user-friendly graphical front-end and various built-in

capability for linking the tools together (feed-forward and feed-back links), performing trade

studies, and collecting and analyzing data from a large number of executions of the integrated

software tools. The interface between ModelCenter® and each software tool is defined by a

wrapper file. The wrapper file contains code instructing ModelCenter® how to transfer

input/outputs and execute the integrated software tool. A wrapper was created for each of the cost

and risk tools contained within I-RaCM. Creation of a wrapper is generally time intensive, so for

many of the tools rather than create a wrapper specific to a fixed set of inputs and outputs, code

was written to automatically generate a wrapper for many alternative instances of the tools. This

approach greatly simplifies setup of I-RaCM essentially allowing for an integrated model to be

easily created for any application that the toolset is capable of analyzing.

B. NASA Air Force Cost Model (NAFCOM) and Remix

Tool Name: NASA Air Force Cost Model (NAFCOM)

Function: Non-Recurring Development Costs and Recurring Production Costs

Platform: Windows® or Macintosh® Executable

Origination: NAFCOM is a government-sponsored cost model with copyright held by Science Applications

International Corporation (SAIC)

Description: NAFCOM estimates non-recurring costs including Design, Development, Testing, and Evaluation

(DDT&E) and Theoretical First Unit (TFU) costs by employing Cost Estimating relationships

(CERs) derived from the actual costs of various historical systems. Several CER methodologies

are available and required inputs can include common weight breakdown structure masses,

programmatic complexities, and subsystem-specific variables (duration, volume, electrical power,

structural efficiency, etc.). Programmatic costs are also estimated including system test hardware,

Table 1: Complete List of I-RaCM Tools

I-RaCM Function Tool(s) / Methodology

CURRENT TOOL SET

Integration Environment ModelCenter® by Phoenix Integration

Nonrecurring Cost (DDT&E and TFU) NAFCOM via the Remix Wrapper Generator

SEER-H by Galorath, Inc. via the Remix Wrapper Generator

Facilities and Equipment Cost Facilities and GSE Operations Assessment (FGOA) Tool

Reliability Reliability_Calc Tool Event Sequence Diagrams and Fault Trees

Campaign Profile Definition Campaign Manager

Technology Effects Technology Impact Matrix

Output visualization and exploration Stack’em

Probabilistic Design Analysis ProbWorks by PiBlue Software

Optimization OptWorks by PiBlue Software

PARTIALLY INTEGRATED TOOLS

Technology Investment Technology Cost Estimator (TCE) Prototype

Operations Simulation Descartes Discrete Event Simulation in Arena®

Business Case and Market Evaluation Cost and Business Analysis Module (CABAM)

Nodal Economic Space Commerce Tool (NESC)

FUTURE TOOLS

Risk/reliability and Consequences Probabilistic Risk Assessment

Availability and Performability Stochastic Petri Nets, Markov Models

Analysis Data Sharing Portable Visualizer


6

integration, assembly, & checkout, system test operations, ground support equipment, systems

engineering & integration, and program management. The Remix software, described below,

provides a means for ModelCenter® to interface with the GUI-only NAFCOM.

Tool Name: Remix

Function: Interface NAFCOM with ModelCenter®


Origination: Developed by SpaceWorks Engineering and commercially marketed by SpaceWorks Software

Description: Remix provides a means for ModelCenter® to interface with the GUI-only NAFCOM by

automatically generating a wrapper file for any unique NAFCOM model and by controlling the

NAFCOM GUI at runtime. NAFCOM does not have command line or simple text file run modes,

so the GUI controller effectively “clicks” on the controls to execute the necessary functions of the

program. The current version of Remix is capable of generating wrappers that provide

ModelCenter® interfaces for all conventional cost estimating equation (CER) input variables,

conventional CER complexity factor inputs, complexity generator (CG) CER universal factors,

and CG subsystem-specific input variables (e.g. structural efficiency, design lifetime, volume,

output power). In addition, Remix-generated wrappers include the Liquid Rocket Engine Cost

Module (LRECM) submodel to NAFCOM. LRECM was developed by Boeing’s Rocketdyne

division (now Pratt & Whitney Rocketdyne) and provides an alternative liquid rocket engine cost

estimating methodology with input variables such as thrust, chamber pressure, propellant type,

engine cycle, and process improvement factors. Systems Integration input variables are not yet

included in Remix. Output variables (DDT&E, STH, FU, DD, and Production cost) for all

subsystems and Systems Integration are included. Remix takes only a matter of seconds to

generate wrappers for NAFCOM models to be run either with or without “Risk,” NAFCOM’s

terminology for uncertainty.

C. SEER-H and Remix Tool Name: SEER-H

Function: Non-Recurring Development Costs, Recurring Production Costs, Operations and Support Costs


Origination: SEER-H is a commercial software product available from Galorath, Inc.

Description: SEER-H estimates the costs of development, production, operations, and support for mechanical,

electronic, structural, and hydraulic hardware elements. CERs underlying the cost estimates are

derived from the actual costs of various hardware components. Knowledge Bases are available to

select appropriate cost analogies for each system component and to adjust variables defining

mission, program, development environment, and production environment cost factors. Cost

outputs can be obtained either as point value estimates or probabilistically. Programmatic costs are

also estimated including system test hardware, integration, assembly, & checkout, system test

operations, ground support equipment, systems engineering & integration, and program

management. The Remix software, described below, provides a means for ModelCenter® to

interface with SEER-H.

Tool Name: Remix

Function: Interface SEER-H with ModelCenter®


Origination: Developed by SpaceWorks Engineering and commercially marketed by SpaceWorks Software

Description: Remix provides a means for ModelCenter® to interface with SEER-H by automatically generating

a wrapper file for any unique SEER-H model. Remix takes advantage of SEER-H’s Server Mode

to execute SEER-H in conjunction with a semaphore and command file. The current version of

Remix is capable of generating wrappers that provide ModelCenter® interfaces for nearly all

SEER-H project, roll-up, mechanical work element, electronic work element, operations and

support, site, and add-in parameters. System Level Analysis parameters are also included in the

wrapper generated by Remix. Knowledge base and SEER-H calculated values for inputs are

preserved by the wrapper. Output variables (DDT&E, Production, Operations, and other costs) for

all subsystems and Systems Integration are included. Remix takes only a matter of seconds to

generate wrappers for SEER-H models to be run either deterministically or probabilistically.


7

D. Facilities and Ground Operations Assessment (FGOA) Tool

Tool Name: Facilities and Ground Support Equipment Operations Assessment (FGOA)

Function: Ground and Infrastructure Costs

Platform: MS Excel®

Origination: FGOA was previously developed by SpaceWorks Engineering with assistance from Construction

Cost Consultants, Incorporated in 2004 under contract to NASA Kennedy Space Center

Description: FGOA is a spreadsheet tool used to calculate and aggregate launch vehicle infrastructure costs and

inflate them to desired year dollars. NASA KSC periodically publishes its Real Property Database

and other technical databases of GSE costs. These databases are used to update and calibrate

FGOA. SpaceWorks Engineering adjusts the estimates as necessary to account for variances from

the historical estimates (e.g. unusual costs of copper wiring in today’s market, variances in the

costs of steel and supplies, new regulations and requirements, changes in labor costs).

E. Reliability_Calc

Tool Name: Reliability_Calc

Function: Reliability Estimation


Origination: SpaceWorks Engineering internally developed reliability analysis tool

Description: Reliability_Calc uses a combination of Fault Tree Analysis (FTA) and Event Sequence Diagrams

(ESD) to quantify top-level reliability metrics such as Loss of Mission (LOM) and Loss of Crew

(LOC) for the complete system or architecture being modeled. FTA is used to find failure rates of

given designs based on a bottoms-up reliability failure analysis. For an architecture analysis, an

FTA is developed for each transportation element of the architecture. These individual FTA results

are then combined into an overall architecture level FTA to determine loss of mission results. The

component failure rates determined by FTA are also combined into an ESD which determines the

overall architecture crew survival rates. The ESD uses the results from several FTAs as well as

success rates associated with various mitigation scenarios in order to determine crew survival.

Monte Carlo analysis is also integrated into the reliability assessment tool to provide mission

success and crew survival rates that meet a given certainty level. Reliability_Calc contains a

macro for the automatic generation of a ModelCenter® wrapper for this tool such that it can be

integrated within I-RaCM. This automatic wrapper generation capability eases the burden on the

user when new elements or events are added to the FTA. The user can simply run the macro to

generate a wrapper for the current set of inputs and outputs, regardless of the system for which the

fault trees and event sequence diagrams have been customized.

F. Technology Cost Estimator (TCE)

Tool Name: Technology Cost Estimator (TCE)

Function: Early Technology Development Costs


Origination: A prototype of this tool has been developed by SpaceWorks Engineering

Description: TCE provides estimates for new technology development by relating a simple set of inputs to an

underlying database of historical technologies. The model is designed to estimate the cost for each

Technology Readiness Level (TRL) step-change (e.g. from 1 to 2 or 4 to 5) up to a TRL of 6. A

concise set of qualitative and quantitative user inputs such as current TRL, length of the research

activity, degree of funding availability, and extent of revolutionary innovation define the

characteristics of the candidate technology. The cost estimate is made possible in part by a

database of historical technology development costs. The technologies in the database ranging

across several technology categories including avionics, manufacturing, operations, propulsion,

structure/materials, subsystems, and thermal protection systems (TPS) may be selected by the user

for inclusion or exclusion in the estimating equation. The current model is in beta form (version

0.6) and has the general qualities of the final model along with a database of approximately forty

technologies in the database.


8

G. Campaign Manager

Tool Name: Campaign Manager

Function: Define the Desired Missions to be Conducted Over the Campaign Lifetime


Origination: This tool was developed by SpaceWorks Engineering

Description: The campaign manager allows the user to define the desired mission schedule for various elements

of an aerospace architecture. The campaign start year and number of elements to be utilized in

each year of the campaign are input in tabular form. The campaign can span up to 16 years in the

current version of the campaign manager. A ModelCenter® wrapper was also created for the

campaign manager for its integration into I-RaCM. The I-RaCM user can easily evaluate the

effects of various campaigns on life cycle cost and reliability by inputting different mission

schedules.

H. Technology Impact Matrix

Tool Name: Technology Impact Matrix

Function: Allocate Impacts of Developed Technologies Amongst Inputs to Other Tools



Description: A technology impact matrix (TIM) is a matrix of available technologies versus relevant input

variables of other tools in the integrated model. The matrix is populated with the absolute setting

or multiplicative factor for those input values which are affected when each technology is active or

inactive. The technology impacts captured in the TIM have some positive or negative effect on

performance, cost, or reliability metrics. The Technology Impact Matrix maps the use of a given

technology to its impact on other design variables such that these metrics can be quantified.

Application of a Technology Impact Matrix in the I-RaCM environment has the potential to

provide a more complete picture of the true cost versus benefit of employing a particular

technology. In a typical conceptual design environment, a TIM may be used to appropriately

reflect the impact of technologies on performance values. The development cost of these

technologies may also be taken into account, but the use of a technology may also have cost

implications for the development and production of the element to which the technology is

applied. For example, the use of composite propellant tanks has the advantage of lower tank mass,

but may also have implications for manufacturing and design integration. The use of a TIM in a

multidisciplinary framework like I-RaCM allows for these primary and secondary aspects of

technology development to be taken into account.

I. Stack’em

Tool Name: Stack’em

Function: Program Phasing, Output Visualization and Exploration



Description: The purpose of Stack’em is to collect, post-processes, and graphically summarize key outputs

from the upstream analyses in I-RaCM. Cost, reliability, and risk (uncertainty distribution

parameters) generated by NAFCOM, FGOA, TCE, and other tools are input to Stack’em where

they are evaluated over the program life cycle. Design and Development (DD) costs are spread

over time using user input variables defining beta curves, DD phase duration, pre-phase A cost as

a percentage of DD cost, pre-phase A duration, and sustaining engineering cost as a percentage of

DD cost. System Test Hardware and up to five planned tests can be defined to calculate Test and

Evaluation (T&E) costs. Production costs over the life of the program take into account learning

curve effects, unit production rates, and the campaign mission model. The user can also input

program budget and overhead wraps data such as projected budget values by year, a percentage

above total cost to hold as reserves, a number of years to shift this reserve if desired, and the

percentage of total cost to charge to the program for full cost wraps. Visual graphics produced by

Stack’em aid the user in rapidly evaluating the outputs of I-RaCM and in performing trade studies.

A “sand chart” depicting the cost roll-up is provided, along with other charts showing costs by

category (DD, T&E, production, and technology maturation), a Gantt chart style timeline for these

categories, two reliability breakdown charts, and several others. Stack’em has built-in trade space


9

exploration features such as the ability to optimize program schedule in order to meet budget

constraints. A ModelCenter® wrapper necessarily complements Stack’em for integration with the

other tools, and the Stack’em tool has a macro that can automatically generate this wrapper.

J. Descartes

Tool Name: Descartes9

Function: Operations Simulation

Platform: Windows® Executable combined with MS Excel®

Origination: Descartes was developed by SpaceWorks Engineering and is implemented within Rockwell

Automation’s commercially available Arena® software

Description: Descartes is a Discrete Event Simulation (DES) approach to operations analysis of aerospace

systems in Rockwell Automation’s Arena® software. Descartes can be employed to simulate the

operations of applications such as lunar surface systems, launch vehicles, and in-space supply

chains. Descartes-Hyperport is one implementation of Descartes currently in development for the

analysis of advanced reusable launch vehicle operations. Microsoft Excel® and VBA interfaces

enable integration of Descartes-Hyperport into I-RaCM. Inputs for the analysis model include

descriptions of the basic vehicle concept of operations, mission models, and performance

information. Typical outputs from Descrates include turn-around time, operations costs, number

and specialties of personnel, and equipment needs.

K. Nodal Economic Space Commerce (NESC) Tool

Tool Name: Nodal Economic Space Commerce (NESC) Tool10,11,12

Function: Commercial Business Case Evaluation and Market Simulation

Platform: Java Executable

Origination: Developed by SpaceWorks Engineering

Description: The NESC model is a dynamic space market simulation and financial engineering tool that uses

Agent-Based Modeling (ABM) techniques to simulate the complex interactions between supply

and demand. ABM has been used in the past to represent plants and animals in ecosystems,

vehicles in traffic, and autonomous characters in animation and games. A wide range of

organizations including Macy's, Humana, and the U.S. Air Force Space Command have used and

benefited from ABM simulations. Agent-based economic simulation is of higher fidelity than

spreadsheet models, and better reflects reality by expressly modeling the individual actions and

interactions of companies, their customers, and their competitors. NESC enables the simulation of

price competition, new entrants to the market, reliability effects, and product differentiation.

Various types of NESC model inputs (required economic return, vehicle capability, costs, etc.)

define the worldview and behaviors of entities in the model. Each company “agent” autonomously

decides its pricing strategy given market conditions and limited competitor information, together

with its own unique costs and product characteristics. Exercising NESC probabilistically is a

necessity since each run results in a different outcome depending on the decisions of the agents.

Probabilistic simulation also allows for investigation of reliability effects. NESC outputs the

financial health of each company (cash flows, Net Present Value, market share, etc.) and can be

used to explore various scenarios including supply vs. demand effects, customer influences, price

changes over time, and product differentiation. NESC has previously been employed by

SpaceWorks Engineering to analyze the International Space Station transportation services market

and the sub-orbital space tourism market.

L. Cost and Business Analysis Module (CABAM)

Tool Name: Cost and Business Analysis Module (CABAM)13

Function: Commercial Business Case Evaluation

Platform: Excel®

Origination: Initially developed at the Georgia Institute of Technology then further developed and modified by

SpaceWorks Engineering

Description: CABAM determines the cash flows and other financial measures of success for a hypothetical

company given programmatic, cost, and market demand inputs. Program definition inputs include

fleet size, flight rates, and program duration. The model takes a corporate finance mentality as far

as economic modeling and requires certain inputs such as financial ratios and rates (debt-to-equity


10

ratio, discount rates, etc.) to define the financial structure of the hypothetical company. Revenues

are calculated using this information along with nonrecurring costs, recurring costs, and market

demand elasticity. Government incentives can be modeled in the form of direct cash contributions,

tax-breaks, and loan guarantees. Outputs include the Net Present Value of the company, breakeven

point, internal rate of return, and price sensitivity information. Year-by-year detailed output

information is presented in traditional income, balance sheet, and cash flow statements.

M. ProbWorks

Tool Name: ProbWorks for ModelCenter®

Function: Probabilistic Simulation

Platform: Java Plug-In for ModelCenter®

Origination: Developed and commercially sold by PiBlue Software

Description: A suite of uncertainty and sensitivity analysis tools including Advanced Monte Carlo, Discrete

Probability Optimal Matching Distribution (DPOMD), Pareto Sensitivity, and Response Surface

Equation (RSE) Generator. The probabilistic capabilities in ProbWorks are useful when trying to

minimize the computational expense for Monte Carlo simulations, ranking the influence of input

variables, and design space approximation through Response Surface Methodologies (RSM). As a

ModelCenter® plug-in component, ProbWorks can be easily set up to perform these actions for

the integrated tools of I-RaCM. The inputs and outputs selected for a ProbWorks simulation can

be any combination of inputs and outputs from any tool in I-RaCM, giving the user unlimited

flexibility to investigate uncertainty and sensitivity of the model.

N. OptWorks Tool Name: OptWorks for ModelCenter®

Function: Optimization

Platform: Java Plug-In for ModelCenter®

Origination: Developed and commercially sold by PiBlue Software

Description: OptWorks is a suite of non-gradient-based optimization tools for use within Phoenix Integration's

ModelCenter® collaborative design framework. Each driver component has a particular benefit or

utility to different classes of problems. The individual drivers in the current release include

Genetic Algorithm (GA), Simulated Annealing (SA), Coordinate Pattern Search (with or without a

Compass Search option), Grid Search, Random Walk, and Random Search. The optimization

algorithms in the OptWorks suite are useful for a wide variety of complex optimization problems

in which the design space may be non-smooth or discontinuous. Within I-RaCM, OptWorks

allows for optimization of the global model. For example, the total program cost can be optimized

subject to schedule constraints by exploring the values of multiple technical inputs to the cost

models.

III. Lunar Exploration Architecture Example Case Studies

An implementation of I-RaCM consisting of a selection of the available tools was created to demonstrate the

basic capabilities and usefulness of the integrated model. Four case studies were conducted. The departure point for

all case studies was a baseline lunar exploration architecture in which no new technologies are required and a

moderate flight schedule is assumed. The first case study demonstrates the schedule optimization capability of

Stack’em whereby the schedule is automatically adjusted to meet a given budget. In the second case study, an

alternate architecture scenario in which four new technologies are developed, is compared with the baseline

architecture. The third case study compares a campaign with a more aggressive flight schedule to the baseline.

Lastly, a Monte Carlo simulation on the proxy performance outputs is conducted for the final case study yielding

probability distributions on the cost of the baseline architecture.

A. Baseline Architecture Overview

The modeled architecture draws heavily on the Exploration Systems Architecture Study (ESAS) released by

NASA in 2005 with the addition of a habitat, an unpressurized rover, and other surface system elements. The

purpose of the ESAS activity was to explore options for NASA’s initiative to return to the Moon.14

I-RaCM

modeling focused on the cost and reliability aspects of the in-space transportation and lunar surface elements

apportioned to lunar exploration. The intention was to isolate the lunar campaign from other possible associated


11

activities such as International Space Station support. The ESAS architecture is a “1.5 launch” approach which

incorporates Earth Orbit Rendezvous (EOR) en-route to the moon and Lunar Orbit Rendezvous (LOR) on the return

leg. All propulsive stages in this architecture use conventional chemical liquid rocket engines.

Earth to orbit launch for crew and cargo is accomplished using Space Shuttle-derived systems. The four

astronauts ride to orbit aboard the Crew Exploration Vehicle (CEV), a combination of an Apollo-like Crew Module

(CM) capsule and a Service Module (SM). The CEV is launched atop the Crew Launch Vehicle (CLV) which

consists of a Space Shuttle Solid Rocket Booster (SRB), and a newly-designed upperstage. Meanwhile, a second

launch vehicle, referred to as the Cargo Launch Vehicle (CaLV), boosts the Earth Departure Stage (EDS) and Lunar

Surface Access Module (LSAM) into LEO. Once in LEO, the EDS, LSAM, and CEV rendezvous and the EDS

performs a trans-lunar injection (TLI) burn to place the entire stack on a transfer orbit to the moon. The empty EDS

is then jettisoned.

When the LSAM and CEV arrive in the vicinity of the moon, the LSAM performs a lunar orbit insertion (LOI)

burn. The LSAM separates from the CEV in Low Lunar Orbit (LLO) and descends to the surface of the moon with

all four crew members aboard. Upon the conclusion of the surface stay, the ascent stage of the LSAM lifts off and

returns to LLO to rendezvous with the waiting CEV. The LSAM ascent stage is discarded, and the CEV SM

performs a Trans-Earth Injection (TEI) burn to place the vehicle on a return path to Earth.

As the CEV approaches Earth, the SM is jettisoned and the CEV CM enters the atmosphere directly. The blunt-

body capsule decelerates aerodynamically in the upper atmosphere, deploys parachutes once an appropriate Mach

number is reached, and eventually executes a land or water landing.

In addition to the primary transportation elements of this architecture described above, the case studies included

three additional elements. These were a surface habitat, an unpressurized rover, and a catch-all category for lunar

surface systems including extravehicular activity (EVA) elements ranging from pressure suits to tools, surface

power units, and lunar communications surface units. The LSAM is assumed to be designed to carry a substantial

payload to the lunar surface in addition to the human crew. This capability will enable delivery of items such as

rovers and habitats. Figure 2 summarizes the baseline architecture elements and some of their key characteristics.

The baseline lunar exploration program begins in 2007 and runs for 21 years until 2027. Optional technology

development, turned on and off through the TCE component, begins in 2007 as well. The first human lunar mission

is flown in 2018 and the full campaign runs until program end in 2027. The first mission is a lunar fly-by only and

does not include a lunar landing. After the fly-by, there is one lunar landing mission a year for the next three years.

During this time, two habitat modules, an unpressurized rover, and some surface systems equipment is delivered.

Two missions a year are conducted from that point forward until program end. Two additional habitats and three

additional rovers are landed by 2025, along with additional surface systems equipment needed for power and

communications. Sixteen total missions are conducted in the ten year campaign period.


12

Figure 2. Summary of elements of case study lunar exploration architecture

Includes

new in-space pressure

suit, EVA suit, and

various EVA tools and mobility aids.

Enables crew

to safely traverse approximately

10-15 km from the landing

site.

Designed to

support 4 crew members for

30 days on the lunar

surface.

Designed to

transport 4 crew members to

and from the lunar surface

from low lunar orbit (LLO).

Uses

LOX/CH4 propellants. Performs

propulsion, thermal

control, and power functions for the CEV.

Designed for

4 crew on lunar mission.

Reusable blunt-body

capsule with ablative TPS.

Uses

LOX/LH2 propellants.

Shuttle-

derived vehicle using 2 SRBs and

a LOX/LH2 core stage.

Shuttle-

derived vehicle using solid rocket

booster (SRB) plus

LOX/LH2 upperstage.

Other

201520142015201620152016201420152014IOC

201020102008200820132013200720072007DD Start Year

-7,100 lbs9,457 lbs8,812 lbs17,712 lbs42,450 lbs637,465 lbs226,650 lbsDry Mass

1

Lunar Surface Access Module (LSAM)

1

Surface Habitat

2

UnpressurizedRover

2

Lunar Crew Exploration

Vehicle (CEV) Crew Module (CM)

2


Vehicle (CEV)

Service Module (SM)

2---Pre-Phase A Duration

EVA and Surface Systems

Earth Departure

Stage (EDS)

Cargo Launch Vehicle (CaLV)

Crew Launch Vehicle (CLV)

Element Name

Includes

new in-space pressure

suit, EVA suit, and

various EVA tools and mobility aids.

Enables crew

to safely traverse approximately

10-15 km from the landing

site.

Designed to

support 4 crew members for

30 days on the lunar

surface.

Designed to

transport 4 crew members to

and from the lunar surface

from low lunar orbit (LLO).

Uses

LOX/CH4 propellants. Performs

propulsion, thermal

control, and power functions for the CEV.

Designed for

4 crew on lunar mission.

Reusable blunt-body

capsule with ablative TPS.

Uses

LOX/LH2 propellants.

Shuttle-

derived vehicle using 2 SRBs and

a LOX/LH2 core stage.

Shuttle-

derived vehicle using solid rocket

booster (SRB) plus

LOX/LH2 upperstage.

Other

201520142015201620152016201420152014IOC

201020102008200820132013200720072007DD Start Year

-7,100 lbs9,457 lbs8,812 lbs17,712 lbs42,450 lbs637,465 lbs226,650 lbsDry Mass

1

Lunar Surface Access Module (LSAM)

1

Surface Habitat

2

UnpressurizedRover

2


Vehicle (CEV) Crew Module (CM)

2


Vehicle (CEV)

Service Module (SM)

2---Pre-Phase A Duration

EVA and Surface Systems

Earth Departure

Stage (EDS)

Cargo Launch Vehicle (CaLV)

Crew Launch Vehicle (CLV)

Element Name


13

B. I-RaCM Tools for Case Studies

The selection, integration, and linking of I-RaCM tools depends on the specific architecture being modeled and

goals of the analysis. The pre-existing wrappers and ModelCenter® Link Editor allow the tools to be quickly

integrated. A subset of all available I-RaCM tools was selected to model the lunar exploration architecture in order

to demonstrate general functionality without overly complicating the discussion. The organization, relationship, and

flow between tools would likely be similar for other architectures. The tools integrated in this example

implementation of I-RaCM are the Campaign Manager, NAFCOM, FGOA, Reliability_Calc, TCE, TIM, Stack’em,

and ProbWorks. In addition, it was necessary to include one additional component to serve as a proxy for the

performance loop. For this example implementation, it was decided not to combine I-RaCM with traditional

conceptual design sizing and performance components (e.g. propulsion, mass estimating, aerodynamics, etc.).

However, a cost estimating process requires the outputs of the performance analysis as inputs, so it was necessary to

create a proxy for the performance loop. The Performance and Sizing Proxy is simply a spreadsheet of

representative performance analysis outputs to serve as inputs to the downstream cost and reliability estimating

tools. Values defining technical specifications for elements of the lunar exploration architecture such as mass,

power, volume, efficiency, and duration are entered into the spreadsheet. While the variable values do not change as

a result of an integrated performance loop, changes to performance outputs can be imitated by changing their values

within the spreadsheet.

Figure 3 is a screenshot of the complete I-RaCM in ModelCenter®. The topmost component is the Performance

and Sizing Proxy. The outputs from this component feed into NAFCOM cost models for elements of the lunar

architecture. The TCE tool informs the Technology Impact Matrix which technologies are active, and the

Technology Impact Matrix in turn adjusts the values of inputs to the cost and reliability tools. Reliability Calc passes

its calculated reliability outputs to Stack’em. The Campaign Manager, at right, provides mission model information

to Stack’em. Stack’em also collects cost outputs from the NAFCOM components, the FGOA components, and TCE.

Surrounding the entire I-RaCM integrated tool suite is a ProbWorks© Monte Carlo driver which is linked to the

inputs of the Performance and Sizing Proxy and the outputs of Stack’em in order to conduct probabilistic studies on

the performance inputs.

Figure 3: I-RaCM in ModelCenter®


14

The costs of three of the nine architecture elements are modeled explicitly in the I-RaCM prototype. Integrated

NAFCOM components are in this implementation of I-RaCM for the EDS, LSAM, and Habitat. Representative

performance loop outputs from the Performance and Sizing Proxy feed the appropriate NAFCOM component for

these elements. The NAFCOM outputs are then in turn passed to Stack’em. Elements not explicitly modeled with

NAFCOM components have fixed development and production cost values inputted directly into Stack’em. There is

no technology development in the baseline architecture and thus no technology cost.

The inputs to the I-RaCM model of the selected architecture were set to isolate the lunar campaign costs. Thus

the production cost of all elements is taken into account, but it is assumed that there is no development cost for the

launch vehicles. This is reflective of the real world situation where the CaLV and CLV are being developed in part

to service the International Space Station (ISS) and provide Earth-to-orbit heavy lift capability. Development of the

CEV Command Module is also assumed to be offset by development of that spacecraft for crew delivery to the ISS.

Thus the cost inputs to I-RaCM are only those additional costs needed to modify the ISS version of the CEV CM to

the lunar CEV CM. Similarly, facilities and ground equipment cost for the CaLV and CLV is considered to be only

any modification cost to the existing facilities in order to launch the lunar missions. There are two instances of

FGOA in the lunar exploration architecture I-RaCM prototype, one for the CaLV and one for the CLV. The inputs

to each of these FGOA models have been set to account for only the modification costs.

Reliability of a lunar mission depends on all elements functioning properly, so all elements are included in

reliability determination. The Reliability Calc tool has fault tree and event sequence diagram logic for the CaLV,

EDS, CLV, LSAM, Habitat, CEV SM, and CEV CM. Reliability of the unpressurized rover and other surface

systems is not calculated within the Reliability Calc component, but a placeholder with a fixed value is included

within Stack’em.

C. Baseline Architecture Results

While all cost inputs and other data analysis data was solely derived from publicly available sources, the cost

results published herein have been normalized to avoid insinuation that any sensitive information is presented.

Figure 4, a “sand chart” generated by Stack’em, depicts a breakdown of the baseline architecture costs by element

over the duration of the campaign. The red line on the chart depicts the program budget, a year by year dollar

amount entered into Stack’em. With the notional budget shown, it is clear that the program violates the allocated

budget slightly in 2007 through 2009 and more dramatically from 2014 through 2016.

A Gantt chart is available within Stack’em to help the user visualize the schedule for each program element. The

Gantt chart for the baseline architecture and campaign is given in Figure 5. The schedule shows the duration of three

major phases of the product lifecycle; design and development, test and evaluation, and production.

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

2007

2009

2011

2013

2015

2017

2019

2021

2023

2025

2027

No

rma

lize

d Y

ea

rly

Co

st

an

d B

ud

ge

t Program Wrap

Reserves

Facilities & Operations

Technology Maturation

Other Surf. Sys.

Unpress Rover

Hab

LSAM

CEV SM

CEV CM

EDS

CaLV

CLV

Budget

Figure 4: “Sand chart” of total program costs for baseline architecture and campaign


15

Stack’em also outputs a chart of total cost broken down by category. Figure 6 shows this chart for the baseline

lunar exploration architecture. As can be seen, facilities and GSE cost makes up only a small portion of total cost in

the early years of the program. This is due to the relatively small cost of modifying the assumed existing CLV and

CaLV facilities to accommodate lunar missions. Design and Development (DD) cost is primarily apportioned

through the earlier years of the program and does not account for as much of the total cost as does production. DD

dollars after 2017 are due to sustaining engineering cost. Notice that production begins prior to the first mission in

2018. More information about the production schedule can be gleaned from the inventory chart.

The inventory chart within Stack’em shows the user how many units of each element have been produced at any

given time in the life of the program. Given their start date and production rates, elements may build up a small

queue over time. Fractional units can be produced in a given year, accounting for non-whole numbers in the

inventory chart. For the baseline scenario, as shown in Figure 7, several of the elements reach an inventory of four

or more units. Constraints may be set by the user within Stack’em, limiting the total number of units which may be

held in inventory, but such constraints were not set for this analysis.

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

2007

2009

2011

2013

2015

2017

2018

2020

2022

2024

2026

No

rma

lize

d Y

ea

rly

Co

st

by

Ca

teg

ory

Technology

Facilities

Production

T&E

DD

Figure 6: Costs by category for baseline architecture and campaign

2007 2009 2011 2013 2015 2017 2019 2021 2023 2025 2027

CLV

CaLV

EDS

CEV CM

CEV SM

LSAM

Hab

Unpress Rover

Other Surf.

Design and Dev Test and Eval Production

Figure 5: Gantt chart for baseline architecture and campaign


16

Within Stack’em the user can optionally run, via a macro button, a built-in Monte Carlo driver (ProbWorks by

PiBlue Software) to conduct a probabilistic analysis on the element costs. The Monte Carlo simulation samples from

distributions on the input costs of the elements to probabilistically sum the total program cost. The results of this

simulation provide the user with a distribution of total program cost where the variability is due to uncertainty in the

cost inputs. A graphical chart also displays the distributions of cost for each element over the duration of the

campaign, providing the user with information about which elements contribute the most to the uncertainty in total

program cost.

For the current version of Stack’em, the Monte Carlo simulation can be set up automatically using a macro, but

the format of the output chart must be set up mostly manually. The inputs to Stack’em defining the cost distributions

may be triangular, normal, uniform, exponential, Weibull, lognormal, or beta. These cost distribution parameters

may be passed directly from the cost estimating tools (when they are run probabilistically) or input by the user.

For the baseline lunar exploration architecture, triangular distributions were placed on the input costs. The

distributions used were notional for the purpose of illustration. Figure 8 shows the resulting cost distributions. The

minimum and maximum of the output distribution for each element is represented by the bounds of the bar. Points

near the mean of the distribution appear as a darker blue, while the tails of the distribution are lighter in color. The

orange horizontal lines in the bars represent a particular user-specified percentile of the distribution, in this case the

70th

percentile. At a glance the user can see that the LSAM is the most costly element and also one of the most

uncertain. The CLV and CaLV costs are also more uncertain than the remaining elements. The location of the 70th

percentile and mean for the LSAM and CaLV distributions indicate that these distributions are left skewed. As such,

there is a greater probability that their costs could be toward the low side, but also a higher limit to the potential cost

of that element.

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

CLV

CaL

VED

S

CEV

CM

CEV

SM

LSAM

Hab

Unp

ress

Rov

er

Oth

er S

urf.

Sys.

No

rmalized

To

tal C

ost

Dis

trib

uti

on

s

an

d 7

0th

Perc

en

tile

Figure 8: Total cost distributions for Architecture elements

0

1

2

3

4

5

6

7

2012 2015 2018 2021 2024 2027

Nu

mb

er

of

Un

its in

Inven

tory

at

Year

En

d

CLV

CaLV

EDS

CEV CM

CEV SM

LSAM

Habitat

Unpress. Rover

Other Surf. Sys.

Figure 7: Inventory of produced elements over campaign lifetime


17

The mean total program cost from the Stack’em Monte Carlo analysis was three percent higher than the

deterministic point estimate. The 70th

percentile value for total program cost is also available from the Monte Carlo

simulation within Stack’em. These values provide a more robust estimate of cost to the analyst by accounting for

uncertainty in the input estimates.

Loss of mission (LOM) and loss of crew (LOC) calculations made by the Reliability Calc tool are compiled

within Stack’em. Reliability calculation of the baseline architecture considered Lunar Orbit Rendezvous (LOR) and

Earth Orbit Rendezvous (EOR) maneuver reliabilities in addition to the element reliabilities shown in Figure 9. The

LSAM/Hab and CEV are the greatest contributors to failure probability. Loss of crew probability is also calculated

on a per mission basis and then program lifetime reliabilities are calculated within Stack’em from the individual

LOM and LOC estimates.

D. Case Study 1: Program Expenditure Optimization

Recall that the cost of the baseline architecture did not fit under the notional budget curve. The total available

budget, however, well exceeds the total cost of the program. This suggests that the schedule may be adjusted to fit

the budget curve. The Coordinate Pattern Search optimizer of the PiBlue Software OptWorks© Excel® plug-in has

been set up within Stack’em to automatically fit the schedule to the available budget. Running the optimizer yielded

the “sand chart” of Figure 10 after about two minutes. The budget total is nearly the same (not exactly due to

sustaining engineering costs), but it now fits under the budget in all years. Comparing the optimized results to the

previous non-optimized results, the strategy for meeting the budget can be seen. It involves, among other changes,

delaying and condensing production of the CLV and CaLV, producing the surface systems earlier in time, and more

evenly distributing the Habitat development and production costs over time.

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

2007

2009

2011

2013

2015

2017

2019

2021

2023

2025

2027

No

rma

lize

d Y

earl

y C

os

t a

nd

Bu

dg

et Program Wrap

Reserves



Other Surf. Sys.

Unpress Rover

Hab

LSAM

CEV SM

CEV CM

EDS

CaLV

CLV

Budget

Figure 10: Optimized “Sand chart” of total program costs for baseline architecture and campaign

0% 20% 40% 60% 80% 100%

LOM

CLV

CaLV & EDS

LSAM & Hab

CEV-SM & CM

Unpress Rover

EVA

Figure 9: Contribution of elements to a single LOM event for the baseline architecture


18

Optimization of the schedule will not always result in a program that fits under the budget curve. Insufficient

available budget or constraints on how much the schedule of each element can shift may prevent from meeting the

budget. Optimization will, however, almost always result in a program that better fits the budget as compared to the

baseline. The post-optimization results can then be evaluated to identify potential solutions to complete the program.

Examples of potential solutions include adjustments to the campaign, reduction in the technical requirements,

targeted increases in the budget, and element design changes to decrease individual element costs. The integrated

environment of I-RaCM allows for diverse types of potential solutions to be quickly evaluated.

E. Case Study 2: Technology Infusion

Four technologies were selected for the technology infusion case study; composite propellant tanks, high thrust

to weight engine, high density batteries, and integrated vehicle health monitoring (IVHM). These technologies were

intended to be somewhat generic with the intention of demonstrating I-RaCM rather than producing comprehensive

technology application results for a lunar exploration architecture. The technology impact matrix for these

technologies is shown in Table 2. When technologies are switched on within the TCE, the inputs passed to

NAFCOM components from the performance loop proxy and other NAFCOM variable values are adjusted by

means of the Technology Impact Matrix. The adjustment could be in the form of a multiplicative factor or a literal

variable setting. The technologies may also adjust the reliability calculations of Reliability_Calc. The cost of the

technology development itself computed by the TCE is passed to Stack’em.

Table 2: Technology Impact Matrix for Technology Infusion Case Study

Impacted Variable Technology Impact (Tech Off / Tech On)

Element Subsystem WBS Item or Variable

Composite

Propellant

Tanks

High

T/W

Engine

High

Density

Batteries

IVHM

Tank Structural Mass Multiplier -- / -6% Structures

Degree of New Design Setting 5 / 6

Engine Mass Multiplier -- / -25%

DD Complexity Value 0.3 / 0.4

EDS Liquid

Rocket

Engine Unit Complexity Value 1 / 1.25

EPD Mass Multiplier -- / -10%

Storage Capacity Multiplier -- / +5% Habitat Electrical

Power Manufacturing Methods Setting 2 / 1


Storage Capacity Multiplier -- / +5% Electrical


LOM Engine Value .0026 /

.0015

LOM Power Systems Value .001 /

.0007 Reliability

LOM RCS Value .0004 /

.0002

CCDH Mass Multiplier -- / +10%


LSAM

Ascent

Stage

CCDH

Eng. Management Setting 2 / 4

Tank Structural Mass Multiplier -- / -6% Propulsion



Storage Capacity Multiplier -- / +5% Electrical


LOM Engine Value .000022 /

.000015

LSAM

Descent

Stage

Reliability

LOM Power Systems Value .0000395

/ .00002


19

When technologies are added to an architecture, their development costs show up on the cost by category chart

within Stack’em. As Figure 11 shows, technology development costs make only a small contribution to the total cost

of the program for the case study. However, the cost to develop the technologies is only part of the complete

technology cost. The use of new technologies can have implications for the development and production of the

various elements as well, and these differences in element cost are modeled by the interconnected tools within I-

RaCM.

The cost differences due to technology infusion versus the baseline are given in Table 3. For the LSAM and EDS

there is a modest increase in cost with the new technologies due to mass and complexity differences. The Habitat is

nearly the same cost with and without the technologies applied. For the entire program, the addition of technologies

increases cost over the life of the program by 2.3 percent due to the cost of the technology development and the

increased cost of the elements themselves.

In addition to its cost implications, the benefits of the IVHM technology on reliability are captured by I-RaCM.

The IVHM system yields a modest improvement in reliability of the LSAM due to increases in reliability of the

Ascent Stage engine, power systems, and reaction control system; and the Descent stage engine and power systems.

Over the 16 flight lifetime of the campaign, the reliability improvements are more pronounced decreasing the

lifetime LOC events by just over 11 percent.

F. Case Study 3: Baseline Versus Aggressive Campaign

For this case study, a more aggressive mission campaign was entered into the Campaign Manager component

and I-RaCM was run to determine the cost comparison of this campaign to the baseline. The alternative, more

aggressive, campaign begins in 2018 and has the same duration (10 years) as the baseline. The alternative campaign

has a lunar landing scheduled in 2018 instead of just a lunar fly-by, plans two flights per years starting in 2020

rather than 2022, and goes to three flights per year for four years beginning in 2024. A total of 22 missions are on

the manifest compared to 16. The more aggressive campaign also lands five unpressurized rovers instead of four,

and lands additional surface system equipment.

Table 3: Cost Results of Technology Infusion Versus the Baseline Case

Campaign Element Total Cost Over Campaign

With Technology Development

EDS + 5.1 %

LSAM + 3.8 %

Habitat - 0.2 %

Program Total + 2.3 %

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

2007

2009

2011

2013

2015

2017

2018

2020

2022

2024

2026

No

rma

lize

d Y

ea

rly

Co

st

by

Ca

teg

ory

Technology

Facilities

Production

T&E

DD

Figure 11: Costs by category for architecture with technology development illustrating additional

technology development costs in early years


20

The total cost of the aggressive campaign is about twenty percent more than the baseline campaign. Indeed, as

Figure 12 shows, the aggressive campaign busts the available budget. Iterating with the campaign manager and

Stack’em schedule optimizer would allow the user to consider alternate campaigns that are more aggressive than the

baseline campaign that might still fit under the budget curve.

G. Case Study 4: Probabilistic Analysis of Performance Variables

A Monte Carlo simulation was performed on the variables input to the I-RaCM cost and reliability tools from the

performance disciplines, represented by the Performance Loop Outputs Proxy, for this last case study. Probabilistic

simulation in this manner has value in that it accounts for uncertainty in the physical design of the architecture

elements and provides an understanding of the resulting distribution of cost due to this uncertainty. The Monte Carlo

simulation is driven within I-RaCM by the PiBlue ProbWorks© Monte Carlo driver plug-in for ModelCenter®.

A histogram of total program cost for the baseline lunar exploration program is shown in Figure 13. The x-axis

on this chart would normally be presented in billions of dollars, but again has been normalized to completely avoid

any potential confusion about data sources. Much information of value can be gleaned from the shape and percentile

values of the distribution. This particular distribution is fairly tight and centered about the mean, indicating only a

small amount of uncertainty in the cost estimate.

0

5

10

15

20

25

30

35

40

0.992 0.994 0.996 0.997 0.999 1.001 1.003 1.004 1.006

Normalized Total Program Cost

Fre

qu

en

cy

Mean

Figure 13: Histogram of program total cost results from Monte Carlo simulation on performance variables

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

1.1

2007

2009

2011

2013

2015

2017

2019

2021

2023

2025

2027

Yea

rly C

os

t an

d B

ud

ge

t ($

M)

Program Wrap

Reserves



Other Surf. Sys.

Unpress Rover

Hab

LSAM

CEV SM

CEV CM

EDS

CaLV

CLV

Budget

Figure 12: “Sand chart” of total program costs for alternative aggressive architecture and campaign


21

Histograms of element total cost over the length of the program for the EDS, LSAM, and Habitat are shown in

Figure 14. The mean cost of these individual elements is also very close to the deterministic values. This is likely

due to the triangular distributions placed on the performance parameters which were evenly distributed. In more

complex studies, these distributions may not be evenly distributed or the combinatorial effects of variable

interactions could result in non-centered output distributions.

Two Monte Carlo simulations have been discussed in the case studies section of this report and there is an

important distinction between the two worth reiterating. The Monte Carlo driver on the performance output

variables addresses the uncertainty in the performance parameters (mass, power, thrust, etc.), while the Monte Carlo

driver internal to Stack’em addresses the uncertainty in cost estimates. These two simulations could be combined to

address both types of uncertainty simultaneously. In such a scenario, the internal Stack’em Monte Carlo simulation

would be run once for every trial of the external Monte Carlo simulation on the performance parameters.

IV. Summary

The Integrated Risk and Cost Model meets the need for rapid and comprehensive cost and risk assessment early

in the design of a new system. I-RaCM integrates a diverse set of new and industry-standard tools for analysis of life

cycle cost, operations, reliability modeling, and estimation of technology development costs. I-RaCM contains

several novel tools and capabilities that enhance cost/risk insight and facilitate rapid trade space investigation. The

Remix Wrapper Generator provides a previously unavailable capability to integrate the NAFCOM and SEER-H cost

tools in the conceptual design process via ModelCenter®. The Technology Cost Estimator prototype demonstrates

the technical feasibility and usefulness of an early technology development cost estimation tool. Stack’em provides

a means for analysts to view key outputs and provides functionality for optimization of program expenditures and

probabilistic evaluation of program cost estimates.

Case studies conducted of a modern-day lunar exploration architecture demonstrated the functionality of I-

RaCM and the novel capabilities of the integrated tools. The case studies demonstrated basic cost and reliability

analysis, program expenditure optimization, technology infusion, campaign excursions, and analysis of performance

0

10

20

30

40

50

60

70

80

0.962 0.978 0.995 1.012 1.028 1.045

EDS Normalized Total Cost

Fre

qu

en

cy

Mean

0

10

20

30

40

50

60

0.982 0.990 0.998 1.005 1.013 1.021

LSAM Normalized Total Cost

Fre

qu

en

cy

Mean

0

10

20

30

40

50

60

70

0.979 0.988 0.996 1.004 1.013 1.021

Habitat Normalized Total Cost

Fre

qu

en

cy

Mean

Figure 14: Histograms of total cost over the duration of the baseline lunar exploration program for the EDS,

LSAM, and Habitat elements. Results from Monte Carlo simulation on performance variables


22

variable uncertainty on cost outputs. I-RaCM is a continuing development, evolving to include more analysis tools

and visualization capability, with plans in place for more advanced reliability analysis, availability and performance

tools, and discrete event simulation of operations processing. With these upgrades, I-RaCM will continue to improve

conceptual design outcomes.

V. Acknowledgements

The authors would like to thank NASA personnel and others who have contributed to the development of I-

RaCM. Initial conception and development of I-RaCM was conducted by SpaceWorks Engineering in fulfillment of

contract requirements for a 2006 Phase I NASA SBIR. Sponsorship and financial support was provided through this

SBIR award, contract number NNM07AA49C. The authors are grateful to the NASA Contract Officer’s Technical

Representative, Mr. Dan O’Neil at NASA Marshall Space Flight Center, whose shared experience with multi-

disciplinary modeling continues to guide development and use of I-RaCM. The authors would also like to

acknowledge Mr. J.D. Reeves of NASA Langley Research Center for review of a prototype version of I-RaCM and

useful feedback. Technical and organizational project support was provided by Mr. Jon Wallace and Dr. John Olds

of SpaceWorks Engineering, Inc. The authors are thankful for the contributions of these two individuals and the

entire SpaceWorks Engineering staff.

References 1Bradford, J. E., Eklund, D. R., Charania, A., Wallace, J. G., "Quicksat: A Two-Stage to Orbit Reusable Launch Vehicle

Utilizing Air-Breathing Propulsion for Responsive Space Access," AIAA-2004-5950, Space 2004 Conference and Exhibit, San

Diego, California, September 28-30, 2004. 2Bradford, J. E., Eklund, D., "Automated Hypersonic Launch Vehicle Design Using ModelCenter," GT-SSEC-B.1, First

Annual Georgia Institute of Technology Space Systems Engineering Conference, Atlanta, Georgia, November 8-10, 2005. 3Bradford, J. E., Olds, J. R., Wallace, J. G., "A Two-Stage Spacelift Concept Using Air-Breathing Propulsion and Falcon-

Derived Hardware," 54th Joint Army-Navy-NASA-Air Force (JANNAF) Propulsion Meeting (JPM) and 5th Modeling and

Simulation / 3rd Liquid Propulsion / 2nd Spacecraft Propulsion Joint Subcommittee Meeting, Denver, Colorado, May 14-18,

2007. 4St. Germain, B., Olds, J. R., Bradford, J. E., Charania, A., DePasquale, D., Schaffer, M., and Wallace, J. G., "Utilizing Lunar

Architecture Transportation Elements for Mars Exploration," AIAA-2007-6136, Space 2007, Long Beach, California, September

18 - 20, 2007. 5Charania, A., Crocker, A., Olds, J., Bradford, J., "A Method For Strategic Technology Investment Prioritization For

Advanced Space Transportation Systems," IAF-01-U.2.06, 52nd International Astronautical Congress, Toulouse, France, October

1-5, 2001. 6Charania, A., Olds, J.,"Application of the Abbreviated Technology Identification, Evaluation, and Selection (ATIES)

Methodology to a Mars Orbit Basing (MOB) Solar Clipper Architecture," IAC-02-U.5.01, 53rd International Astronautical

Congress, The World Space Congress - 2002, Houston, Texas, October 10-19, 2002. 7Crocker, A., Charania, A., Olds, J., "An Introduction to the ROSETTA Modeling Process for Advanced Space

Transportation Technology Investment," AIAA-2001-4625, Space 2001 Conference and Exposition, Albuquerque, NM, August

28-30, 2001. 8St. Germain, B., Charania, A., Olds, J. R., "A Stochastic Process for Prioritizing Lunar Exploration Technologies," AIAA-

2005-6607, Space 2005, Long Beach, California, August 30 - September 1, 2005. 9DePasquale, D., Kelly, M., Charania, A., Olds, J. R. "Activity-Based Simulation of Future Launch Vehicle Ground

Operations," AIAA-2008-7660, Space 2008, San Diego, California, September 9 - September 11, 2008. 10Charania, A., DePasquale, J., "Simulating the Dynamic Marketplace: An Introduction to the Nodal Economic Space

Commerce (NESC) Model," AIAA-2005-6617, Space 2005, Long Beach, California, August 30 - September 1, 2005. 11DePasquale, D., Charania, A., "Winning in the Next Space Market: Prospects for Financial Success of Commercial

Transportation Services to the International Space Station (ISS)," SEI Case Study, January 2007. 12DePasquale, D., Charania, A., Olds, J. R., "Sub-Orbital Space Tourism: Predictions of the Future Marketplace Using Agent-

Based Modeling," IAC-06-E3.4, 57th International Astronautical Congress, Valencia, Spain, October 2-6, 2006. 13Charania, A., Olds, J. R., "Optimization of a Future RLV Business Case Using Multiple Strategic Market Prices," IAC-02-

IAA.1.1.07, 53rd International Astronautical Congress, The World Space Congress - 2002, Houston, Texas, October 10-19, 2002. 14National Aeronautics and Space Administration, “NASA’s Exploration Systems Architecture Study,” Final Report, NASA-

TM-2005-214062, November 2005.

I-RaCM: A Fully Integrated Risk and Life Cycle Cost Model · framework, providing for a seamless integrated analysis solution. Both newly developed and existing industry-standard

Documents