SURVEY OF ERROR PROPAGATION IN SYSTEMS.

! _ .. - - - - . . _ _ _ . . . . _ . _ . . _ _ . . . - . .- .. . -_. . - - . . . . ______ _ _

.

;

|

,

NUREG/CR 2839ORNL/CSD/TM-190

UCC=NDy,

,

!

I' ;

i

UNION t

CARBIDE Survey of Error

Propagation in Systemsi

|

| V. R. R. UppuluriW. Kuo

?

I ,

,

9

|

6

:

Prepared for the*

} U.S. Nuclear Regulatory CommissionOffice of Nuclear Regulatory Research

Under Interagency Agreement DOE 40-550-75

!t

! 8306010086 830430PDR NUREOCR-2839 R PDR

i

- - - - _ _ - _- . - - _ - . - . , , - . . _ _ - , . , , . _ _ , - _ ,

V

.

.

Printed in the United States of America. Available fromNational Technical Information Service

U.S. Department of Commerce5285 Port Royal Road, Springfield, Virginia 22161

Available from

GPO Sales Program

Division of Technical Information and Document ControlU.S. Nuclear Regulatory Commission

Washington, D.C. 20555

.

This report was prepared as an account of work sponsored by an agency of theUnited States G overnment Neither the U nited S tates Government nor any agency -

thereof, nor any of their employees, makes any warranty, express or implied, or

,assumes any legal liability or responsibility for the accuracy, ccmpleteness, orusefulness of any information, apparatus product, or process disclosed, orrepresents that its use would not infringe privately owned rights Reference hereinto any specific commercial product, r,rocess, or service by trade name, trademark,manufacturer. or otherwise, does not necessarily constitute or imply itsendorsement. recommendatien. or favoring by the United States Government orany agency thereof The views and opinions of authors expressed herein do notnecessarily state or reflect those of theUnited States Government or any agencythereof.

,

4

e

_ _ _

e Ae.a- J - - 4+4e #- e. 5 -4 -4 m .a + A _A- *

b

i

NUREG/CR-2839~ 0RNL/CSD/TM-190Dist. Category RGj

SURVEY OF ERROR PROPAGATION IN SYSTEMS,,

V. R.'R. UppuluriMathematics and Statistics' Research

i

W. Kuoi .

. ENGINEERING PHYSICS-DIVISION-

i -

*

{. Manuscript Completed: March 4, 1983 '

Date Published: April ~1983*

!'

.

-'*,

.

Prepared for the4

U.S. Nuclear Regulatory Commissionj Officelof Nuclear Regulatory Research

Washington, D.C. 205554

Under Interagency Agreenent' D0E 40-550-75

1

-NRC FIN No. 80444

COMPUTER SCIENCESat

j Oak Ridge National LaboratoryPost Office Box Y

Oak Ridge, Tennessee 37830

IUnion Carbide Corporation, Nuclear Division

operating the,

? 4 Oak Ridge Gaseous Diffusion Plant Oak Ridge National Laboratory*

i,

Oak Ridge Y-12 Plant Paducah Gaseous Diffusion Plant*

under Contract No. W-7405-eng-26-

i for the| DEPARTMENT OF ENERGY

:

f

- , - ,,-r -m er,-, --a . - - ..e--,.-me -. - , , - , - - . - . ,_..,,e ,, -,-- .- - - - - n-

.

SURVEY OF ERROR PROPAGATION IN SYSTEMS.

V..R. R. Uppuluri'.

Way Kuo

"

ABSTRACT

Error propagation analysis in reliable systems has been studiedwidely. Unlike classical sensitivity analysis which investigates therange of system performance, the distribution function of systemperformance studies error propagation analysis. Thus, errorpropagation analysis is essentially a statistical analysis.

This paper reviews and classifies current research articles inerror propagation. An overview is presented of error propagationapplied to various systems and models. A standard analysis procedureis also given. Finally, several conclusions are drawn. It isrecommended that i) basic research on error propagation be carriedout, 11) an efficient (least cost) method be developed to analyzelarge-size problems, and iii) human error be included in the system ,modeling. It is our opinion that error propagation analysis betreated as part of decision-making procedures in system analysis.

Error propagation analysis is extremely important for expensive*

or rare-event systems. This report can benefit those who analyzethese systems.

,

Keywords: Error Propagation, Systen Performance

.

$

|

a

9

.

1

F

2

1. INTRODUCTION

An analyst is often faced with predicting the performance of a *

system' prior to its construction and use. The complexity of nuclearpower plants, for example, has led to concern by public and scientificresearchers. Various technioues for reliability evaluation of complex -

systems have been _ studied widely. A recent survey by Hwang, Tillman,and Lee [19] reviews several of these reliability evaluationtechniques. For low failure rate nuclear systems, logic diagrams havebeen used extensively [13].

Whether we are concerned about a nuclear system or not, thesystem failure rate evaluation, although important, may not be theunique consideration :n decision making. An analysis in the designstage generally is nEcessary to determine whether the reliability codeis structurally stable or whether any physical or empiricalconstraints can be exceeded [18]. The effect on system performance(reliability is one kind of performance measure) of uncertainties inits components is also of interest. The sources of theseuncertainties include the possibilities that i) the model used hasbeen incorrectly specified, ii) the correct values of the componentsare not known with confidence, iii) the system performance isevaluated differently due to the change of environment conditions, andiv) that human error is heavily involved in various stages. .

System performance uncertainties contributed by its componentshave been termed " propagation of uncertainties" or, equivalently, .

" error (variance) propagation in systems." Other terminologies suchas " imprecision analysis," " tolerance analysis," and " function ofrandom variables" have also been adopted. in various studies. The termused in this report is error propagation analysis, which is differentfrom classical sensitivity analysis. In classical sensitivityanalysis of a system, we are interested in evaluating the effect onsystem performance of variations in its component's specifications.In error propagation analysis, however, we are interested indetermining the range of system performance, given the range of thecomponent's specifications. Both component and system performancemeasures are treated as random variables. Therefore, errorpropagation is a statistical estimation problem, while classicalsensitivity analysis is an algebraic one. In addition, whileclassical sensitivity analysis has been studied widely ([17,39] havegood discussions on the subject), error propagation analysis has not.

Error propagation analysis is also different from perturbationtheory, which is well known in physics. Perturbation theory discussesthe stability of physical phenomena under the condition that variablesof interest are perturbed by small values. For error propagation,

,

however, we investigate the variation in the distributions of bothcomponent's errors and output performance.

.

The objective of error propagation analysis is to estimate systemperformance variation resulting from components characterized by

3,

random variables. This report surveys recent work in errorpropagation, classifies and reviews articles and reports, sets upanalysis procedures, outlines application problems, and suggests,

future investigations. The systems we are considering here are verygeneral. However, nuclear systems are emphasized because the-inducedtop-event error is of primary interest to scientists in nuclear safetyanalysis. The analysis procedures are also applicable to many systemsas long as the variation of output performance is important. - Systemsother than nuclear problems include, for exanple, long-term economicalplans and modern telecommunication systems.

2. PROBLEMS

Suppose that a system performance, Y, is a function of itscomponent performance, Xj, i=1,2,...,n, through

'

Y = f(X ,X ,...,Xn). (1)1 2

In equation (1), Y may be regarded as system reliability ) [2,21,42],(unreliability) [19], system availability (unavailabilityproduct of availability and reliability [38], physical measurementssuch as cross-section evaluations [12,14], biological measurementssuch as dose measurements [35], management information systen such as

analysis [ profit prediction [34], top-event performance in fault-treelong-term

13,32], or any performance measure of interest. Similar-

discussions _ are also employed to X , i=1,2,... ,n. The function fiin equation (1) either can be in analytical or empirical functionalform. The function f is formulated by various system structures (such*

as network, fault-tree, or complex system configuration), economicalor physical linkages, biological flow path, or computer code. Thenumber of components involved in the construction of f is denoted by n.

For error propagation analysis, we assume that X 's are randomivariables with analytical or empirical distribution functions. As afunction of X 's, Y is also a random variable. Typically n is ailarge number.

In error propagation analysis, a typical problem is what thevariation of Y would be, given the uncertainties of Xj,i=1,2,...,n. Specifically, we may want to determine the followingquantities;

i) Pr[Y>c c, a constant],ii) probab .lity density function (pdf) of Y, or

iii) confidence interval of Y.

Essentially, Y is treated as a random variable. Note that in-

classical reliability analysis, Y is no more than an undeterminedconstant.

.

A second problem in error propagation is to evaluate the relativeimportance of X 's. An important Xi has a great effect on theidistribution of Y, instead of en Y itself. For example, the Xj, for

;

,

i

.

4

all i, may have large variations and be an important attribute on Y,but cause little variation in Y given a significant change of X ; ihence the Xi is of negligible importance. On the other hand, small .

variation of Xi may cause significant variation in Y, and this Xjwill rank highly even if it plays a small role in constructing f.

Since, for'every Xi and Xj, i#j, they may as well be s-dependent -

as s-independent and n is usually a large number, ranking X 's isipertinent and presents a problem.

Given the maximum allowed Y-variation, a third problemconfronting decision makers is how to specify tolerate variation ofX 's by economically utilizing current knowledge on X 's. In thei idesign stage, this is of utmost importance. No solution to thisproblem has yet been found.

3. ANALYSIS PROCEDURES AND STATISTICAL MODELS

To approach the problems presented in-the previous section, fourbasic analysis procedures are recommended as follows:

i) model setup,ii) screening of X 's, i=1,2,... ,n, and leaving only. thosei

important X 's, i=1,2,...,N, where hopefully N << n,iiii) application of statistical design models,~andiv) construction of the pdf for the calculated consequence. '

In certain situations where n is small, then step li) is often skipped[15,23,38]. It is also common that steps 11) and iii) are combined in

,

the analysis. For a typical example of the combined analysis, see Cox[7]. When a Monte Carlo simulation is selected, steps iii) and iv)are sometimes also combined [7,8]. In all cases, construction of thepdf and its associated consequences seems to be the goal forinferences made on Y.

Various statistical models in the analysis of error propagationhave been adopted for the previous steps. These models are outlinedand classified below.

Model Setup

As the first step to investigate the error propagation in asystem, model setup includes construction of the function f 'andestablishment of the uncertainties of X 's. Both of these tasksirequire a thorough understanding of the system. In addition,

extensive experience and theoretical judgement' are equally importantfor model setup. Selection of f is different from system to systemand is called system modeling. Given function f, the uncertainties of '

Xj's have been subjectively assigned by many researchers, (forexample, the assignment of lognormal distribution [34]).

,

Human error is largely involved in the model setup due to theuncertainties of f selection and assignment of components'

_ _ __ _ __ __________ _ _.

5

variations. To simplify our discussions, the following steps arebased on a well set-up model.

*

Screening

Screening involves selection of important variables and deletion,

of unimportant ones. Today, the most reliable - but least elegant -screening procedure remains the direct method [29]. This methodselects as the important variables the X 's whose variations produceithe largest change of Y. This steepest ascent approach ismathematically written as

Important X. = large {BY } , for all i (2)1 X. BX.

1 1

for the continuous case, or

Important X. = large {AY } , for all i (3)1 X AX

$ j

for the discrete case. *

This approach in taking the relative variation of BY/BXi (orAY/AX ) depends on both the location and the distribution of Xj.i.

Because of the number of computer runs required, the direct method iscostly for large n and/or steep ascents of f at varied locations ofX 's.i,

Several recent methods attempt to determine the importance ofXj's for a large n using fewer computer runs. One of them is thematrix approach. Let b denote the vector of sensitivity coefficientswith respect to the n components. The solution to a set of linearequations ,

b=X1AY (4)

leads to the determination of the important components. In equation(4), AY is the variation vector containing changes in Y for N computerruns. The disadvantages of this approach are i) a solution is not

; guaranteed, and ii) one may end up solving an ill-conditioned lineari system. *

A third screening mettfbd applies statistical regressiontechniques. Through a stepwise correlation analysis (backward orforward regression), the important components - to the Y value - areselected. Again, the method is costly and inefficient for large n.

,

The most promising fast-running screening method is the adjointmethod. In this approach, the function f is no longer treated as a.

'black box, as in the previous three screening approaches. 'Thisapproach allows the exact sensitivity coefficient to be evaluated.

;

$

-.

"S

'6

While one run of'the' previous screening methods give.the sensitivityof all responses-to one parameter, one runiof the adjoint method-gives'-

the sensitivities of one. response to all , parameters. . This method .is. V

still under development and hard1to follow.* These four screening approaches are compared in Table 1. -

. Table ~1oClassification of Various' Screening Methods

In The Study of Error Propagation_

_ __ __

Method Coments References-

Direct Method Give exact importance 3, 5, 29, 26ranking of.. independentproperties of X 's butivery expensive for large n.

o* Matrix Approach Deals with a highly.

1, 12, 14, 21undetermined system of -linear equations.

Statistical Importance ranking 4is not 29. .

Regression uniquely determined.Extremely expensive andinefficient for large n. c

. Adjoint method. Not easy to follow, but 1, 30, 31requires fewer runs 'andexact-importance rankingis guaranteed.

-- --------

Statistical DesignE

Once screening is done, the proble:n of n components is nowreduced to the one of: N ' components, and presumably N K n. The n - Ncomponents, which are determined to be of negligible importance to the-variation of the top consequence Y, can be treated as constants in thestatistical design. Unless n is a small number or the screeningprocedure has not been employed, n - N variables are to be dealt within the stage of statistical design. Several approaches are available.

First of all, a classical approach has been widely used which_ ,

applies basic ideas of evaluating the cumulation of uncertainties fromXj. . In this approach, variation of Y is formulated through thephysical structure of f. For example, whenever f constitutes simple .

summations, va'riance of Y is the summation of variance'of X 's. Inithe classical approach, standard deviation (or variance), . coefficientof variation, kurtosis, and skewness are regarded as indices of the

_.

7

variations. The biggest disadvantage of this approach results in thefailure of variation evaluation if the physical structure of Y due toX 's, f, is not available. This approach also does not obtaini.

accurate information on the shape of variation. Nevertheless, thismethod gives a preliminary idea on the range of the variation. Manytextbooks (for example, [10,39]) deal with the classical approach in

.

solving engineering problems.

A second approach applies the Taylor series expansion to -determine the mean and variance of Y. Tukey [40], probably the firstone to use this technique in error propagation, approximated f by anrth degree polyne'nial through Taylor series expansion. The lower-order terms (up to first or second order are the popular ones) in theexpansion are used in the calculation. This -approach can, to acertain extent, deal with dependent relationships among X 's.iHowever, because the differentiations of Y with respect to X 's areiencountered, an analytical form of f is usually required. 0nly themean and variance of Y typically can be obtained by this approach.

A third approach applies response surface methodology. Becausethe uncertainty of Y originates- from statistical variations of X 's,iit is essential that the scheme prescribing these variations maximizetheir effects on the calculated Y. With the response surface methodof uncertainty analysis, the perturbations of the components arecarried out according to an experimental design that enables an'

efficient empirical exploration of the response surface. This is aneffective procedure as long as it is augmented by a foldover designand star points [20]. The combination accounts for the linear and'

quadratic effects of the components, as well as those of 2-f actorinteractions between X 's, while requiring a small number oficomputer runs [8].

The Monte Carlo method can also be used- to obtain empiricalresponse surface equation. This method employs computerized syntheticsampling of the component information. Through repeated randomprocedures to generate various component information, a sufficientnumber of responses, Y, can be computed.

The four statistical designs discussed above are classified andlisted in Table 2.

|i

!

.

!*

!

.

t

I

|

|

|

~

i

8 |1

.

Table 2Classification of Various Statistical Designs

Used in the Study of Error Propagation .,

--

Design Comments References .|,

l

Classical Approach Can deal with a limited 5, 10, 8, 14number of situations 34, 35, 39which are restricted bythe functional form of. f.

4

Taylor Series Analytical structure of f 2, 9, 36, 40Approach is necessary, and only

limited information on Y jis obtained. |

1

Response Surface The only method that can 3, 10, 28, 29Method account for the dependent I

relationships amongX 's.i

Monte Carlo Expensive, but can test 7, 13Method the independence of *

X 's.i=-

'

|Distribution of Y

Since Y is a function of random variables X 's, i=1,2,...,N,idistribution of Y derived from the assignment of distributions of !X 's is actually a Bayesian-type prublem. The distribution of Y 1si '

also of interest for the decision makers. Two different approaches I

can be used to construct the distribution of Y.

The most popular approach is to use the moment matchingtechnique: to match mean, variance, skewness, and kurtosis to thoseof some member of a family of density functions, such as the Pearson

ifamily of distributions. The moments may be derived from i)evaluating the mean of the power of the response surface equation overthe density function of all X 's, or ii) direct calculation from theiresults of Monte Carlo simulation. In the first method, thecomponents' variations are obtained most often by assigning analyticaldistributions [28], whereas, in the second method, the empirical

distributions on the components' variations are sufficient. After the -

first four moments are calculated, the histogram of Y can be drawn.The moment matching technique obtains a rough estimate of thedistribution of Y, but does not identify the true distribution. Some

'

moment matching techniques and the selection of the matcheddistribution functions can be referred to in Bowman [4] and McGrath, |et al, [25]. I

.

.

.

9

T' Monte Carlo simulation gives the- distribution functionitselt, e,d evaluates the response surface. To apply simulation togenerate the distribution function, a stratified sampling procedure isrecommended, which will- adequately cover all statistical*

fluctuations. It is always true that simulation provides anindependent statistical check of precision.

,

Cox [7,8] comments that the Monte Carlo simulation produces noreliable way of determining whether any of the components are dominantor more important than others. Furthermore, if a change is made inthe density of. any input, the entire uncertainty analysis must beredone. The response surface method does not suffer thisdisadvantage, because the response surface equation is estimatedindependently of the uncertainty densities.

References dealing with moment matching techniques and MonteCarlo methods are classified in Table 3. Once the distributionfunction of Y is specified, statistical inferences on Y such as thelower or higher confidence interval can be made. On the other hand,even if the distribution function of Y is not specified, conservative

confidence intervals can be obtained from the lower moments of Y.Apostolakis and Lee [2] and Gonzalez-Urdaneta and Cory [15] adoptedChebyshev inequality to determine the conservative confidenceintervals.

.

Table 3Classification 'f Various Approaches to theo

Determination of Y Distribution in the Study of Error Propagation-

Approaches Comnents References

Moment Matching Relatively inexpensive 2,4,5,6,7,Technique and can shape the tails 9, 8, 15, 22,

more accurately. 27, 29, 34, 36,35, 37

Monte Carlo Relatively expensive. 7, 13, 15, 23,

Methods 25, 33, 36

A Hierarchal Structure of Analysis Model

The procedures recommended in error propagation analysis alongwith various statistical models are outlined in Figure 1. Notice thatin Figure 1 screening, statistical design, and determination of thedistribution of Y are not always in separate forms or always in.

sequence. Combinations of some of these procedures are common. Also,the Monte Carlo method can be used at several stages and can becombined with analytical methods whenever necessary. In sunnary, the-

purpose of utilizing and comparing a variety of models is to makestatistical inference on Y taking into account the variations ofX 's and to serve as a tool for the decision maker.i

_ _ _ ..

-

10

MODEL SET UP

1r .-

A SCREENING 1. direct method3' '

2. matrix approach,

3. regression-technique4. adjoint method

b"

i

STATISTICAL DESIGN 1. classical approach2. Taylor series expansion3. response surface method4. Monte Carlo-method

2'

.

p DISTRIBUTION OF Y 1. moment matching technique. due to response surface

method. due to Monte Carlo '

method2. Monte Carlo method

'F,

STATISTICAL INFERENCE ON Y

f

'FDECISION MAKING

:

Fig. 1. - A Hierarchal Structure of Error Propagation Analysis

.

f

.

- . , _ . . _ _ . _ . m _ _

_ . _

ll

:

' 4. COMPUTER CODES

To manually' analyze' error' propagation problems for a large system.

is almost impossible. _ Several computer codes have been developed to.handle the problem. Each of these computer. codes'is designed underspecial demand and for a specific problem, hence all the codes.

available have-limited usage. Some codes are _ listed as follows.

i) Second-Order Error Propagation Code (S0ERP)[9] ;

This code presents the development and use of second-ordererror propagation equations for the first four moments of afunction of independently ~ distributed random variables.

11) 80VNDS[24,2]The code calculates for the propagation of moments when the-<

underlying distributions of components in a tree islognormally distributed,

i i ii) SAMPLE [32]This code uses a Monte Carlo simulation model to evaluate the"

top-event unavailability.

,iv) MARCH / CORRAL [3,44]

* Uncertainty evaluation is programmed in both the computerj codes and the input data that propagate through to the code'

output for a critical path [32]...

v) SCORE [5,8]A linearized procedure is used in SCORE -to approach errorpropagation problems. This computer" code considers thecombination of random variables which consists of asystematic combination of varied components. An empiricaldistribution may be obtained of the top-event performance ina fault tree framework.

P

5. APPLICATIONS

In an expensive or safety-related system, variation of the-

~

top-event performance is undesirable. Nevertheless, in the operating;

stage,. variation of the top-event performance propagated from theuncertainties of its components performance often cannot beprevented. Hence, in the design stage, top-event and componentvariations deserve great concern. Analysis of error _ propagation,,

t which discusses these variations, has been applied to several types ofI problems. Table 4 classifies the problems and their references. The| table does not ' cover all problems and references in this field; but it*

; presents major applications.*

i

!I

w e w m- r- ae+- - -2--- -:v w- ew - n ----m -r, r ,-e- - w, e-w,---- -e<- - - - - - e r- - - e+ y -.e--n, --- - m t-, .

12

Table _4Classification of Major System Problems-

Approached by Error Propagation Analysis,

=

Major System Problem References~

Computer Code Analysis 26, 40

Energy Modeling 1, 31

General Engineering Systems 10, 16, 43, 45, 49

Management Information 34Systems

Nuclear Cross-Section 12, 14Measurement

Nuclear Fault-Tree 2, 5, 13, 48Analysis

.

Nuclear Reactor Analysis 3, 7, 8, 11,14, 29, 30, 41, 46, 50

,

Radiological Assessment 14, 34, 35, 47

System Effectiveness 15, 23, 27, 37 -

Modeling-- __ _ _ ---

6. CONCLUSIONS AND DISCUSSIONS

Although extensive research has been undertaken on errorpropagation in various system modeling problems, great uncertaintystill exists. Current methodolog .s are limited to the determinationof top-event error propagated from the variations of components in apredefined model, 'i.e., f in this study. Even if f is deterministic,current approaches i) do not take care of the dependent situation

economical plans)i's (which is extremely important in long-termamong component X

, ii) do not consider time dependent performance (ina combat mission system, time is probably the key factor to evaluatethe performance), iii) imposes too many assumptions to restrict theerrors incurred in X 's (which have never been justified), and'iv)ihas no discussion upon the influence of error (which propagates into3

top-event performance) to decision makers. In addition, whenever thenumber of components in a system of interest is large, existingmethods to approach error propagation are very costly.

Furthermore, the structure of f is not always known. There .

always exists uncertainty about the model itself, and quite possiblythat uncertainty of f propagates major variation on Y.

, - . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . _

13

In conclusion, much research needs to be done. Some generaldirections are recommended as follows.

'

l. Basic statistical research on error propagation should bestarted so that concrete information about- Y through

* different structures of f and X 's may be obtained.i* Decision theory also needs to be extended to combine theconcrete information with possible loss functions associatedwith the top-event performance.

2. An efficient method is required (at the least cost) toestimate the variation of Y. There is no guarantee that thecurrent screening methods can significantly deleteunimportant X 's. There is a risk that an-important Xiiwill be removed from consideration, and that even -afterscreening the number of components in' a system it is stilllarge. Current screening methods do not provide answers to

.these problems.

3. In practical system modeling, human error may be the biggestattribute to the variation of output performance. Therefore,-a method should be developed to account for human error.Although, analytical form of human error is unknown (and maynever be known), recognition of human error contribution tothe top-event is extremely important.-

7. ACKNOWLEDGEMENTS.

Kuo acknowledges the support from the 1981 Summer FacultyResearch Participation Program of the Oak Ridge National Laboratoryand the Oak Ridge Associated Universities.

.

.

e

- -, -- - , _ _ . . , , _ _ , - _ . _.

14

REFERENCES

.

[1] Alsmiller, R. G., Jr., et al ., Interim Report on Model EvaluationMethodology and the Evaluation of LEAP, Report ORNL/TM-7245,Union Carbide Corporation, Nuclear Division,1980. -

.

[2] Apostolakis, G., and Lee, Y. T., " Methods. for the Estimation ofConfidence Bounds for the Top-event Unavailability of FaultTrees," Nuclear Engineering and Design, Vol. 41, pp. 411-419(1977).

[3] Baybutt, P., and Kurth, R. E., Uncertainty Analysis of LightWater Reactor Meltdown Accident Consequences: MethodologyDevelopment, BATTELLE, Columbus Laboratories, 1978.

[4] Bowman, K.'0., One Aspect of the Statistical Evaluation of aComputer Model, Report ORNL/CSD-52, Union Carbide Corporation,Nuclear Division, 1980.

[5] Colombo, A. G., " Uncertainty Propagation in Fault-tree Analysis,"Synthesis and Analysis Methods for Safety and Reliability Studies

' (Apostolakis, G., Garribba, S., and Volta, G., ed.) N. Y.: PlenumPub., 1980. .

[6] Colombo, A. G., and Jaarsma, R. J., "A Powerful Numerical Methodto Combine Random Variables," to appear as EUR report. .

[7] Cox, N. D., " Comparison of Two Uncertainty Analysis Methods,"Nucl. Sci. Eng., Vol. 64, pp. 258-265 (1977).

[8] Cox, N. D., and Cermak, J. 0., " Uncertainty Analysis of thePerformance of Complex Systems," Energy Sources, Vol. 1, pp.339-359 (1974).

[9] Cox, N. D., and Miller, C.'F., User's Description of Second-orderError Propagation (SOERP) Computer Code for StatisticallyIndependent Variables, Report TREE-1216, Idaho NationalEngineering Laboratory, 1978.

[10] Crandall, K. C., and Seabloom, R. W., Engineering Fundamentals,N. Y.: McGraw-Hill, 1970.

[11] Denning, R. S., Cybulskis, P., Wooten, R. 0., Baybutt, P., andPlummer, A. A., " Methods for the Analysis of Hypothetical ReactorMeltdown Accidents," Proc. ANS Topical Meeting on Thermal Reactor

.

Safety, Idaho, 1977.,

[12] Dragt, J. B. , Dekker, J. W. M. , Gruppelaar, H. , and .

Janssen, A. J., " Methods of Adjustment and Error Evaluation ofNeuton Capture Cross Sections; Application to Fission ProductNuclides," Nucl. Sci. Eng., Vol. 62, pp. 117-129 (1977).

15

[13] . Fault . Tree Handbook, NUREG-0492, U.S. Nuclear RegulatoryCommission, 1981.

.

[14] Gersth, S. A. W., Dudziak, D. J., and Muir, D. W., " Cross-SectionSensitivity and Uncertainty Analysis with Application to a FusionReactor," Nucl. Sci. Eng., Vol. 62, pp. 137-156 (1977)..

[15] Gonzalez-Urdaneta, G. E., and Cory, 8. J., " Variance andApproximate Confidence Limits for Probability and Frequency ofSystem Failure," IEEE Trans. Reliability, Vol. R-27, pp. 289-293(1978).

[16] Herd, G. R., Madison, R. L., and Gottfried, P., "The Uncertaintyof Reliability Assessments," Proc. 10th Natl. Symp. on Rel. andQ.C., Washington, D.C., p. 33-40 (1964).

[17] Hiller, F. S., and Lieberman, G. J., Introduction to OperationsResearch, San Francisco, Holden-Day,1980.

[18] Himmelblau, D. M., and Bischoff, K. B., Process Analysis andSimulation: Deterministic Systems, N. Y.: Wiley, 1968.

[19] Hwang, C. L. , Tillman, F. A. , and Lee, K. H. , "A Review ofComplex System Reliability Evaluation," submitted to IEEE Trans.

'

Reliability, 1980.

[20] Kliejnen, J. P. C., Statistical Techniques in Simulation, ,,

Part II, N. Y. Marcel-Dekker, 1975.

[21] Krieger, T. J., et al., " Statistical Determination of EffectiveVariables in Sensitivity Analysis," Trans. Am. Nuclear Science,Vol. 28, pp. 515 (1977).

[22] Ku, H. H., " Notes on the Use of the Propagation of ErrorFormulas," J. Research Natl. Bur. Stand., pp. 263-273 (1966).

[23] Kuo, Way, " Simulation on Bayesian Availability," 1981 FallORSA/TIMS National Meeting, October 11-14, 1981.

[24] Lee, Y. T., and Apostolakis, G., " Probability Intervals for theTop Event Unavailability of Fault-Trees," Report UCLA-ENG 7663,University of California, Los Angeles,1976.

[25] McGrath, E. J., et al., " Techniques for Efficient Monte CarloSimulation," Vol. 1, Report ORNL-RSJC-38, Oak Ridge NationalLaboratory (1975).

.

[26] McKay, M. D., et al., " Report on the Application of StatisticalTechnique to the Analysis of Computer Codes," Report

*

LA-NUREG-6526-MS, U.S. Nuclear Regulatory Commission (1976).

[27] Murchland, J. D., and Weber, "A Moment Method for the Calculationof a Confidence Interval for the Failure Probability of aSystem," Proc. Annual Rel. & Maint. Sym.,1972.

;

a

t

16

[28] Myers, R. H. ,- Response Surf ace Methodology, N. Y. : Allyn and_

Bacon, 1975..

[29] Nguyen, D. H., "The Uncertainty.in Accident ConsequencesCalculated by Large Codes due to. Uncertainties in Input," Nucl.Tech., Vol . 49, pp. 80-91 (1980) . ,

[30] Oblow, E. M.', " Sensitivity Theory for Reactor Thermal' HydraulicsProblems," Report ORNL/TM-6303, Oak Ridge National Laboratory,

.

1978.

[31] Perey, F. G., Contribution to Screening Methodology, Report.

ORNL-5744, Union Carbide' Corporation, Nuclear Division, 1981.

[32] Reactor Safety Study, An Assessment of Accident ~ Risks in U.S.Commericial Nuclear Power Plants, WASH-1400.(NUREG-74/014),1975.

[33]' Rettig, W. H., Radd, _ M. E., Vesely, W. E., and Ybarrondo, L'. , J. ,"A Monte Carlo Uncertainty Estimate for a Complex System ofEquations,"' Proc. ANS Meeting, Michigan, 1973.~

[34] Schaeffer, D. L., and Hoffman, _F. ' 0., " Uncertainties 'in -Radiological Assessments - A Statistical Analysis of_ RadiciodineTransport via the Pasture-Cow-Milk Pathway," Nuclear. Technology,-Vol. 45, pp. 99-106 (1979). '

[35] Schwarz, G., and Hoffman, F. O'., "Imprec'ision of Dose. Predictions yfor-Radionuclides Released to the Environment: An Application ofa Monte Carlo Simulation Technique,"' Environment International,Vol. 4, (1980).

[36] Smith, D. E., "A Taylor's Theorem - Central Limit Theorem -Approximation: Its Use in Obtaining the Probability Distributionof Long-Range Profit," Management Science, Vol.18, pp. B214-8219(1971).

[37] Thompson, M., " Lower Confidence Limits and' a-Test of Hypothesesfor System Availability," IEEE Trans. Reliability,.Vol. R-15,pp. 32-36 (1966).

[38] Tillman, F. A. , Kuo, Wa~ , and Hwang, .C. L. . *A Numerical .ySimulation of the System Effectiveness - A Renewal TheoryApproach," Proc.1982 Annual Reliability and MaintainabilitySymposium, Los Angeles, pp. 252-261 (1962).

[39] Tomovic', R., and Vukobratovic', M., General Sensitivity Theory, 1

N. Y.: American Elsevier, 1972. '

[40] Tukey, J. W., "The Propagation of Errors, Fluctuations, andTolerances," Statistical Techniques Research Group, Department of '

! Mathematics, Princeton University, 1954.-

.

J

. - - , - - - , - . - ~ . - - , . _--_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _> _-- - - - - - ..

17

[41] Vaurio, J. K., " Reactor Development Program Progress Report,"Report ANL-RDP-76, Argonne National Laboratory,1978.

[42] Vesely, W. E., Reliability Quantification Techniques Used in the-

Rasmussen Study," Reliability and Fault Tree Analysis, pp.775-803 (1976).

.

[43] Watson, I. A., "The Rare Event Dilemma and Common CauseFailures," Proc.1982 Annual Reliability and MaintainabilitySymposium, pp. 5-10 (1982).

[44] Wooten, R. 0., and Avci, H. I., " MARCH Code Description andUser's Manual," Report NUREG/CR-1711, 1980.

[45] Barry, B. A., Errors in Practical Measurement in ScienceEngineering and Technology, N. Y.: Wiley, 1978.

[46] Canali, S., and Olivi, L., "A Statistical Library for theResponse Surface Methodology Application to the Nuclear Safety,"COMPSTAT 1980, pp. 1-7 (1980).

[47] Coldstein, R. A., and Kaolo, F. R., " Biological Risk UncertainAnalysis," presented at Intl. Symp. Energy and Ecol. Modeling,Kentucky, April 20-23, 1981

.

[48] The Kuljian Corporation, Sensitivity Analysis for FaultTree / Event Tree Calculated Risk, Report to EPRI contract No. TSA81-425, 1981.-

[49] Mazumdar, M., Marshall, J. A., and Chay, S. C., " Propagation ofUncertainties of Problems of Structural Reliability," NuclearEngineering and Design, Vol. 50, pp. 163-167 (1978).

[50] Olivi, L., " Response Surface Methodology in Risk Analysis,"! Synthesis and Analysis Methods for Safety and Reliability Studiesj ( Apostolakis, G., Garribba, S., and Volta, G., ed.), N. Y.:

| Plenum Pub., pp. 313-327, 1980.|

|

.

| -

i

:|

|

19

,

NUREG/CR-2839.ORNL/CSD/TM-190 .Dist. Category RG,

'

INTERNAL DISTRIBUTION

1. Central Research Library 12. R. E. Funderlic2. ORNL Patent Section 13. T. L. Hebble3. Y-12 Technical Library -

. 14-18. .W. KuoDocument Reference Section ;19. W. E. Lever

4. Laboratory Records Department - RC 20. A. L. Lotts5-6. Laboratory Records 21. T. J. Mitchell

7. CSD K-25 Library - Reports Section 22. K. E. Shultz/8. J. A. Carpenter . Biometrics Library ~9. H. P. Carter /CSD X-10 Library 23-27. V. R. R. Uppuluri

10. G. F. Flanagan, Jr. 28. R. C. Ward11. E. L. Frome 29. D.-G. Wilson

30. T. Wright

EXTERNAL DISTRIBUTION

31. Dr. Lee Abramson,' Acting-Chief, Applied Statistics, U.S. Nuclear'

Regulatory Commission, Washington, D.C. 20555

32. Dr. Donald M. Austin, Division of Engineering, Mathematical and'

.Geosciences, Office of Basic Energy Sciences, Department of'Energy, ER-15, Germantown Building, Room J-311, Washington, D.C.20545

33. Dr. Richard E. Barlow, Industrial Engineering & OperationsResearch Department, University of California, 3115 EtcheverryHall, Berkeley, California 94720 .

34. Dr. Peter Bloomfield, Department of Statistics, PrincetonUniversity, Princeton, New Jersey 08540

35. Dr. L. Lynn Cleland, Engineering Research Division, LawrenceLivermore National Laboratory, P.O. Box 808, Livermore,California 94550

36. Dr. D. G. Hoel, National Institute of Environmental HealthSciences, P.O. Box 12233, Research Triangle Park, North Carolina27709

;,

37. Dr. James W. Johnson, Division of Risk Analysis, U.S. Nuclear'

Regulatory Commission, Washington, D.C. 20555.

_ __

20

38. Dr. M. A. Kastenbaum, Director of Statistics, The Tobacco-Institute, Inc.,1875 I Street, NW, Suite 800, Washington, D.C.20006 .

39. Dr. Lester Lancaster, Division of Risk Analysis, U.S. NuclearRegulatory Commission, Washington, D.C. 20555 s

40. Ms. Judith A. Mahaffey, Statistics, Systems Department, PacificNorthwest Laboratory, P.O. Box 999, Richland, Washington 99352

41. Dr. F. L. Miller, Jr., Desert Research Institute, 4582 MarylandParkway, Las Vegas, Nevada 89109

42. Dr. George A. Milliken, Department of Statistics, Calvin Hall,Kansas State University, Manhattan, Kansas 66505

43. Dr. P. K. Niyogi, Methodology Development Section, RiskMethodology & Data Branch, Division of Risk Analysis, Office ofNuclear Regulatory Research, U.S. Regulatory Commission,Washington, D.C. 20555

44. Professor S. A. Patil, Department of Mathematics, TennesseeTechnological University, Cookeville, Tennessee 38501

45. Dr. Ronald Peierls, Applied Mathematics Department, Brookhaven ' '

National Laboratory, Upton, New York 11973'

46. Professor Milton Sobel, Department of Mathematics, University ofCalifornia, Santa Barbara, California 93017

47. Dr. William E. Vesely, Batelle Columbus Laboratory, 5055 KingAvenue, Columbus, Ohio 43201

48. Dr. Ray A. Waller, S-1, Statistics, Los Alamos NationalLaboratory, P.O. Box 1663, Los Alamos, New Mexico 87545

49. Dr. Edward J. Wegman, Statistics and Probability Program, Officeof Naval Research, 800 N. Quincy Street, Arlington, Virginia22217

50. Office of the Assistant Manager for Energy Research andDevelopment, Department of Energy, Oak Ridge Operations Office,Oak Ridge, Tennessee 37830-

51-52. U.S. DOE Technical Information Center, Oak Ridge, Tennessee37830

e

53-337. Given Distribution as shown in NRC Category RG, Systems &Reliability Reports (10 copies - NTIS).

_ - _ _ _ . _ - - - _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ - _ _ - _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ -

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . _ __ _ _ . _ . _ . _ _ . _ _ _ _ - - _ - - - - - . - -

h

,

1

A

o

e

.

55502 CO NOTGNIHSAW -105-W

GERUN RDP-RB TGM BUP & YCILOP 'CDIT FO VID MDA

CRN SU67010N 1 778870555021

,

$

,

T

_ _ _ _ _ _ _ _ _ _ _ _ - -