Quantitative assessment with improved fast Fourier transform based method by signal mirroring

ARTICLE IN PRESSG ModelNED-5071; No. of Pages 10

Nuclear Engineering and Design xxx (2008) xxx–xxx

Contents lists available at ScienceDirect

Nuclear Engineering and Design

journa l homepage: www.e lsev ier .com/ locate /nucengdes

Quantitative assessment with improved fast Fourier transform basedmethod by signal mirroring

latedon. Rnds wstigathe t

provemonslso thodsimpr

Andrej Prosek ∗, Matjaz Leskovar, Borut MavkoJozef Stefan Institute, Jamova cesta 39, SI-1000 Ljubljana, Slovenia

a r t i c l e i n f o

Article history:Received 7 March 2007Received in revised form 16 April 2008Accepted 18 April 2008Available online xxx

a b s t r a c t

The comparison of calcuhydraulic code qualificati(FFTBM) favors certain trelast data point of the inveaccuracy calculation viewsstudy was therefore to imsignal mirroring. In the deBesides two case studies aof the Best-Estimate MethThe results show that theunbiased way.

1. Introduction

For years the fast Fourier transform based method (FFTBM),after proposed in 1990 (Ambrosini et al., 1990) has been used for

Please cite this article in press as: Prosek, A., et al., Quantitative assessmmirroring, Nucl Eng Des (2008), doi:10.1016/j.nucengdes.2008.04.012

accuracy quantification of code calculations (D’Auria et al., 1994,1996, D’Auria and Ingegneri, 1997; Mavko et al., 1997; Prosek et al.,2002). The FFTBM shows the measurement–prediction discrepan-cies – accuracy quantification – in the frequency domain. It assistsin answering how to conduct an objective comparison and whereimprovements to the input model are needed. It is also used as astandard tool for nodalization qualification (D’Auria and Giannotti,2000). For single variable comparison also other powerful indicesof accuracy may be used as proposed by Kunz et al. (2002). Furtherimprovements of FFTBM were done in the last years. An FFTBMwith new accuracy measures is described in detail in Prosek andMavko (2003). In the study (Prosek and Mavko, 2004) the capabil-ity to calculate time dependent code accuracy is described. Theseimprovements were used for quantifying code calculations of thelarge break loss of coolant accident (LOCA) in the RD-14M facility(Prosek et al., 2006). In the study by Prosek and Leskovar (2005)it is described how the FFTBM with the capability to calculatetime dependent code accuracy was successfully adapted for theuse within the severe accidents field. The FFTBM was applied tothe severe accident International Standard Problem ISP-46 (Phe-

∗ Corresponding author. Tel.: +386 1 5885450; fax: +386 1 5885377.E-mail address: [email protected] (A. Prosek).

0029-5493/$ – see front matter © 2008 Elsevier B.V. All rights reserved.doi:10.1016/j.nucengdes.2008.04.012

results to experimental measurements is very important for thermal-ecently, it was observed that the fast Fourier transform based methodhen an edge (difference) is present in the signal between the first and the

ted time signal. Namely, the discrete Fourier transform used for the codeime domain signal as an infinite periodic signal. The purpose of the present

the FFTBM regarding the unphysical edge impact. This was achieved bytration it was shown how the improved FFTBM by signal mirroring works.e code accuracy of the LOFT L2-5 test calculations performed in the frame

Uncertainty and Sensitivity Evaluation (BEMUSE) program was assessed.oved FFTBM by signal mirroring judges the accuracy in a consistent and

© 2008 Elsevier B.V. All rights reserved.

bus FPT1) (Leskovar and Mavko, 2006). A new measure for theindication of the time shift between the experimental and the cal-culated signal was proposed. It was also suggested to make alloperations for the method improvement in the time domain (e.g.logarithmic scale, moving average), as it is very difficult to make theadjustments in the frequency domain. In spite of several improve-ments already done, another most important need for improving

ent with improved fast Fourier transform based method by signal

the FFTBM was identified recently. Namely, in the applications ofFFTBM with the capability to calculate time dependent code accu-racy it was observed that the accuracy changes very much whenthe experimental signal sharply increases or decreases (e.g. tri-angular shape). When the signal starts to return to its previousvalue, the accuracy also follows this trend. It is not logical that thestill present discrepancy decreases the accuracy instead of increas-ing it. This problem was hidden in the past when FFTBM wasapplied only to a few time windows and/or intervals. But whenthe time dependent accuracy was calculated by FFTBM, the defi-ciency of the original method became evident. This was recentlyconfirmed when reviewing the results of the previous applica-tions of FFTBM (D’Auria et al., 1994; Prosek et al., 2002). Similarfindings can be obtained also when looking at the recent FFTBMapplication (OECD/NEA, 2006). The original FFTBM gave an unre-alistic judgment of the accuracy for monotonically increasing ordecreasing functions, causing problems in FFTBM results interpre-tation and leading to variable dependent accuracy criteria (D’Auriaet al., 1994). The purpose of the present study was therefore toimprove the FFTBM in this respect. It was found out that the reasonfor such an unrealistic calculated accuracy of increasing/decreasing

dx.doi.org/10.1016/j.nucengdes.2008.04.012

http://www.sciencedirect.com/science/journal/00295493

mailto:[email protected]


ARTICLE ING ModelNED-5071; No. of Pages 10

2 A. Prosek et al. / Nuclear Engineering

Nomenclature

AA average accuracyAAm average amplitude by signal mirroringAAtot total weighted average amplitude
BEMUSE Best-Estimate Methods Uncertainty and Sensitivity
EvaluationCSNI Committee on the Safety of Nuclear InstallationsECC emergency core coolantECCS Emergency Core Cooling SystemFFT fast Fourier transformFFTBM fast Fourier transform based methodFPT fission products testHPIS high-pressure injection systemIAEA International Atomic Energy AgencyISP International Standard ProblemLOCA loss of coolant accidentLOFT loss of fluid test facilityLPIS low-pressure injection systemNEA Nuclear Energy AgencyOECD Organization for Economic Co-operation and Devel-

opmentPWR pressurized water reactorSG steam generator

signals is the so called edge effect. Namely, if the values of the firstand last data point of the investigated signal differ, then there arediscontinuities present in the periodically extended signal seen bythe discrete Fourier transform (DFT), which views the finite domainsignal as an infinite periodic signal. The discontinuities give severalharmonic components in the frequency domain, thus increasing thesum of the amplitudes, on which FFTBM is based, and by this influ-encing the accuracy. The quantitative contribution of the edge effecton the accuracy may be very unpredictable and can overshadow thecontribution of the discrepancies of the compared functions on theaccuracy.

In the presented study, it is proposed how to resolve the problemof the edge effect on a unique way by signal mirroring. A demon-stration application is presented to show that the improved FFTBMby signal mirroring gives a realistic and consistent judgment of theaccuracy also for monotonically increasing or decreasing functions,and for all other signals influenced by the edge effect.


2. FFTBM description

Almost two decades ago the original FFTBM was proposed forquantification of code accuracy (Ambrosini et al., 1990). The FFTBMis systematically described in Prosek et al. (2002). In short, theFFTBM is a method, which shows the measurement–predictiondiscrepancies in the frequency domain. The method purpose is toquantify the accuracy of code calculations based on the amplitudesof the discrete experimental and error signal calculated by the fastFourier transform (FFT).

The digital computers can only work with information that isdiscrete and finite in length. There is no version of the Fourier trans-form that uses finite length signals (Smith, 1999). The way aroundthis is to make the finite data look like an infinite length signal. Thisis done by imaging that the signal has an infinite number of sam-ples on the left and right of the actual points. The imagined samplescan be a duplication of the actual data points. In this case, the sig-nal looks discrete and periodic. This calls for the DFT to be used.There are several ways to calculate DFT. One method is FFT. Whileit produces the same results as the other approaches, it is incredi-

PRESSand Design xxx (2008) xxx–xxx

bly more efficient. The key point to understand the FFTBM is thatthe periodicity is invoked in order to be able to use a mathematicaltool, i.e. the DFT.

2.1. Periodic nature of discrete Fourier transform

The DFT views both, the time domain and the frequency domain,as periodic (Smith, 1999). This can be confusing and inconvenientsince most of signals used for comparison are not periodic. Nev-ertheless, the user must conform to the DFT’s view of the world.Fig. 1 shows two different interpretations of the time domain sig-nal. In the upper figure the time domain is viewed as N points.This represents how digital signals are typically acquired in sci-entific experiments and code calculations. For instance, these 64samples might have been acquired by sampling some parametersat regular intervals of time. Sample 0 is distinct and separate fromsample 63 because they were acquired at different times. From theway this signal was formed, there is no reason to think that thesamples on the left side are even related to the samples on theright.

Unfortunately, the DFT does not see things this way. As shown inthe lower figure, the DFT views these 64 points to be a single periodof an infinitely long periodic signal. This means that the left sideof the acquired signal is connected to the right side of a duplicatesignal. Likewise, the right side of the acquired signal is connected tothe left side of an identical period. This can also be thought of as theright side of the acquired signal wrapping around and connectingto its left side. In this way, sample 63 occurs next to sample 0, justas sample 22 occurs next to sample 23. This is referred to as beingcircular, and is identical to viewing the signal as being periodic.

The most serious consequence of time domain periodicity is theoccurrence of the edge. When the signal spectrum is calculated withDFT, the edge is taken into account, despite the fact that the edge hasno physical meaning for comparison, since it was introduced arti-ficially by the applied numerical method. It is known that the edgeproduces a variegated spectrum of frequencies due to the disconti-nuity of the edge. These frequencies originating from the artificiallyintroduced edge may overshadow the frequency spectrum of theinvestigated signal.

The original FFTBM is based on the sum of the amplitudes ofthe frequency spectrum of the investigated signal. The frequenciesoriginating from the artificially introduced edge may significantlycontribute to the sum of the frequency spectrum amplitudes ofthe investigated signal. Consequently, the accuracy measure based


on the original FFTBM is significantly influenced by the edge andtherefore does not present a consistent accuracy measure of thesignals being compared. This inconvenient drawback of the origi-nal FFTBM may be completely cured by eliminating the artificiallynumerically introduced edge. This may be efficiently done by sig-nal mirroring, where the investigated signal is mirrored before theFFTBM is applied. By composing the original signal and its mirroredsignal (signal mirroring), a symmetric signal with the same charac-teristics is obtained, but without introducing an edge when viewedas an infinite periodic signal.

2.2. Signal mirroring

If we have a function F(t) where 0 ≤ t ≤ Td and Td is the transienttime duration, its mirrored function is defined as Fmir(t) = F(−t),where −Td ≤ t ≤ 0. From these two functions a new function is com-posed which is symmetrical around Td: Fm(t), where 0 ≤ t ≤ 2Td.By composing the original signal and its mirrored signal, a sig-nal without an edge between the first and the last data sampleis obtained when periodically extended, and is called symmetrizedsignal. The symmetrized signal is shown in Fig. 2. The upper fig-



A. Prosek et al. / Nuclear Engineering and Design xxx (2008) xxx–xxx 3

viewe
Fig. 1. Periodicity of the DFT’s time domain original signal. The time domain can besignal, shown in the lower figure.
ure shows the finite length signal and the lower figure shows theinfinite length periodic signal. We see that the symmetrized signalhas no edge also when viewed as a periodic signal. Therefore, in thesum of the frequency spectrum amplitudes only the amplitudes ofthe investigated signal are considered, as it should be. The FFTBMusing the symmetrized signal is called “improved FFTBM by signalmirroring”.


2.3. Average amplitude

For the calculation of measurement–prediction discrepan-cies the experimental signal Fexp(t) and the error signal �F(t)are needed. The error signal in the time domain is defined as�F(t) = Fcal(t) − Fexp(t), where Fcal(t) is the calculated signal. Thecode accuracy quantification for an individual calculated variableis based on the amplitudes of the discrete experimental and errorsignal obtained by FFT at frequencies fn, where n = 0, 1, . . ., 2m and mis the exponent defining the number of points N = 2m+1. The averageamplitude AA is defined:

AA =∑2m

n=0|�F(fn)|∑2m

n=0|Fexp(fn)|, (1)

where |�F(fn)| is the error signal amplitude at frequency fn and|Fexp(fn)| is the experimental signal amplitude at frequency fn. TheAA factor can be considered a sort of average fractional error andthe closer the AA value is to zero, the more accurate is the result.Typical values of AA are from 0 to 1.

d as N samples in length, shown in the upper figure, or as an infinitely long periodic

2.4. Total average amplitude

The overall picture of the accuracy for a given code calculation isobtained by defining average performance index, that is the AAtot

(total accuracy)

AAtot =Nvar∑

(AA)i · (wf)i (2)


i=1

with

Nvar∑

i=1

(wf)i = 1, (3)

where Nvar is the number of the variables analyzed, and (AA)i, and(wf)i are the average amplitude and the weighting factor for the i-thanalyzed variable, respectively. Each (wf)i accounts for the experi-mental accuracy, the safety relevance of particular variables and itsrelevance with respect to the primary pressure. The weights mustremain unchanged during each comparison between code resultsand experimental data concerning the same class of a transient. Formore information on weighting factors refer to (D’Auria et al., 1994;Prosek et al., 2002). The acceptability factor for AAtot was set to 0.4.

2.5. Deficiency of original FFTBM

When the above original accuracy measures were proposed(Ambrosini et al., 1990) it seems that the impact of the edge effectwas not considered. It is evident that this is a deficiency if the accu-



4 A. Prosek et al. / Nuclear Engineering and Design xxx (2008) xxx–xxx

can be
Fig. 2. Periodicity of the DFT’s time domain symmetrized signal. The time domainperiodic signal, shown in the lower figure.
racy measure depends on the unphysical edge resulting from theintrinsic property of the DFT mathematical method, which treatsthe investigated finite length signal as an infinite length periodicsignal. Namely, for the comparison the shape of the finite discretesignal is important and not its edge characteristics. Also the visualcomparison of signals is done in such a way.

It was already mentioned that the periodicity is invoked in order


to use a mathematical tool. Therefore, this influence should beeliminated. This was done by signal mirroring. By composing theoriginal signal and its mirrored signal, a finite signal with the samecharacteristics as the original signal is obtained, but without intro-ducing an edge when periodically extended since the value of thefirst and last data sample is the same. We should keep in mind thatDFT views the time domain as periodic. The composed original andmirrored signal is called symmetrized signal. When DFT is appliedto the finite length symmetrized time domain signal the edge effectis obviously not introduced anymore.

2.6. Calculation of AAm

For the calculation of the average amplitude by signal mirroring(AAm) the Eq. (1) is used as for the calculation of AA, except that,instead of the original signal, the symmetrized signal is used. Thereason to symmetrize the signal was to exclude the edge from thesignal without influencing the characteristics of the investigatedsignal. The signal is automatically symmetrized in the program forthe improved FFTBM by signal mirroring as is described in (Prosekand Leskovar, 2006).

viewed as N samples in length, shown in the upper figure, or as an infinitely long

As already mentioned, the edge has no physical meaning forcomparison, since it was introduced artificially by the appliednumerical method, but FFT produces harmonic componentsbecause of it. By mirroring, the shapes of the experimental and errorsignal are symmetric and their spectra are different from the orig-inal signals spectra, mainly because they are without unphysicaledge frequency components. Due to different spectra the sum of


the amplitudes changes in both, the numerator and the denomina-tor of Eq. (1). To further demonstrate this in Sections 3.2 and 3.3,two new definitions are introduced for the average amplitude of theerror signal (AAerr) and the average amplitude of the experimentalsignal (AAexp), related with the numerator and denominator of Eq.(1):

AAerr = 12m + 1

2m∑

n=0

|�F(fn)|, (4)

AAexp = 12m + 1

2m∑

n=0

|Fexp(fn)|. (5)

It should be noted that also when both, the original and errorsignal are without the edge, in principal different AAerr and AAexp

may be obtained with the original FFTBM and the improved FFTBMby signal mirroring. Indeed AA and AAm are slightly different mea-sures also if the signals are without an edge. The values obtainedwith the original FFTBM and the improved FFTBM by signal mirror-ing are the same only for symmetrical original signals. But this isnot really a deficiency of the proposed improved FFTBM by signal


IN PRESSering and Design xxx (2008) xxx–xxx 5

Table 1Information about participants performing LOFT L2-5 test calculations in the frameof the BEMUSE program

Organization Calculation ID Code used

Commissariat a l’EnergieAtomique (CEA), France

CEA CATHARE 2.5

EDO “Gidropress” (GID), Russia GID TECH-M-97Gesellschaft fur Anlagen- und

Reaktorsicherheit mbH(GRS), Germany

GRS ATHLET 1.2C

ARTICLEG ModelNED-5071; No. of Pages 10

A. Prosek et al. / Nuclear Engine

mirroring, since it is important only that the method judges theaccuracy on a realistic and unbiased way and that it is consistentwithin itself. In Section 3.4, it is presented how the accuracy calcu-lated with the improved FFTBM by signal mirroring can be directlycompared to the accuracy calculated with the original FFTBM.

3. Demonstration application

In this section, some results are shown to see the advantages ofthe improved FFTBM by signal mirroring compared to the originalFFTBM. First the test and the calculations used in the demonstra-tion application of the improved FFTBM by signal mirroring arebriefly described. Then two case studies are presented. The case 1study is presented to show how the edge (when present) alwayschanges the accuracy even if this is logically not expected. In thecase 2 study the accuracy of one variable is calculated for two timewindows. Average amplitudes of the error and experimental sig-nal are calculated to show the impact of the edge effect. Finally,the improved FFTBM by signal mirroring is applied to LOFT L2-5calculations performed in the frame of the Best-Estimate MethodsUncertainty and Sensitivity Evaluation (BEMUSE), Phase II to fur-ther show that the improved FFTBM by signal mirroring is moreconsistent in the quantitative assessment than the original FFTBM.

3.1. Test and calculations description

For the demonstration of the new proposed accuracy measureobtained by FFTBM based on the symmetrized signal, the LOFT L2-5 test was selected. This test was selected for this demonstrationonly since the huge amount of data was available to the authors andsince the assessment of these test results with the original FFTBMwas already published in the literature (OECD/NEA, 2006). The cal-culations were performed in the frame of the BEMUSE program. Inthe phase II of the BEMUSE program, which is the re-analysis of theISP-13 exercise, post-test calculations of the LOFT L2-5 test wereperformed.

The LOFT Integral Test Facility is a scale model of a pressurizedwater reactor (PWR) (Bang et al., 2004). The intent of the facilityis to model the nuclear thermal-hydraulic phenomena that wouldtake place in a PWR during a LOCA. The LOFT Emergency Core Cool-ing System (ECCS) simulated the ECCS of a commercial PWR. Itconsisted of two accumulators, a high-pressure injection system(HPIS), and a low-pressure injection system (LPIS).


The objective of the ISP-13 test was to simulate a LOCA caused bya double-ended, off-shear guillotine cold leg rupture coupled witha loss of off-site power in the nuclear LOFT test facility. Delayedinitiation of the HPIS and the LPIS were assumed.

The experiment was initiated by opening the quick openingblowdown valves in the broken loop hot and cold legs. The reactorscrammed on low pressure at 0.24 s. Following the reactor scram,the operators tripped the primary coolant pumps at 0.94 s. Accu-mulator injection of emergency core coolant (ECC) to the intact loopcold leg began at 16.8 s when the system pressure dropped below4.2 MPa. Delayed ECC injection from the HPIS and LPIS began at23.90 and 37.32 s, respectively. The fuel rod peak cladding temper-ature of 1078 K was attained at 28.47 s. The accumulator emptiedat 49.6 s. The cladding was quenched at 65 s, following the corereflood. The LPIS injection was stopped at 107.1 s, after the experi-ment was considered complete.

Table 1 shows the information on organizations participat-ing, the ID of calculations and the codes used in the phase II ofthe BEMUSE program (OECD/NEA, 2006). In total 14 calculationsfrom 13 organizations were performed using six different codes(nine different code versions). The code most frequently used was

Institut de Radioprotection etde Surete Nucleaire (IRSN),France

IRSN CATHARE 2.5

Japan Nuclear Energy Safety(JNES), Japan

JNES TRAC-P 5.5.2

Korea Atomic Energy ResearchInstitute (KAERI), SouthKorea

KAERI MARS 2.3

Kozponti Fizikai Kutato Intezet(KFKI), Hungary

KFKI ATHLET 2.0A

Korean Institute of NuclearSafety (KINS), South Korea

KINS RELAP5/MOD3.3

Nuclear Research Institute(NRI), Czech Republic

NRI-1 RELAP5/MOD3.3

Nuclear Research Institute(NRI), Czech Republic

NRI-2 ATHLET 2.0A

Paul Scherrer Institute (PSI),Switzerland

PSI TRACE 4.05

Turkiye Atom Enerjisi Kurumu(TAEK), Turkey

TAEK RELAP5/MOD3.3

Universitat Politecnica deCatalunya (UPC), Spain

UPC RELAP5/MOD3.3

University of Pisa (UPI), Italy UPI RELAP5/MOD3.2

RELAP5/MOD3.3. From Table 1 it can be seen also that only one orga-nization performed two calculations using two different codes. Formore detailed information on the calculations the reader is referredto (OECD/NEA, 2006).

3.2. Case 1 study by signal mirroring

To demonstrate how signal mirroring works, FFT was applied tothe signals shown in Figs. 1 and 2 and the AAexp values of signalswere calculated per Eq. (5).

Imagine that you would quantitatively assess two variables, withthe shape of the experimental signals as shown in Figs. 1 and 2.Most probably you would judge that the judgment based on FFTBMshould be the same as the area below the curve is the same when


normalized with the number of data samples (the area below thesymmetrized curve is the double area of the original signal). Never-theless, in the case of the original FFTBM different values of AAexp

are obtained (25.87 for the original signal and 16.29 for the sym-metrized signal), while in the case of the improved FFTBM by signalmirroring the same results are obtained for AAexp of the originaland symmetrized signal (16.29). This means that both, the originalFFTBM and the improved FFTBM by signal mirroring produce thesame results when no edge is present in the signal and when thesignal is symmetrical. This example clearly shows that the originalFFTBM is not consistent when an edge is present in the signal.

The difference between the original FFTBM and the improvedFFTBM by signal mirroring results mainly due to the unphysicaledge introduced by the applied numerical method, and can bedirectly quantified. The AAexp of the symmetrized signal has tobe extracted from the AAexp of the original signal. In our examplethis contribution is 9.58 ((25.87 − 16.29) = 9.58). This means thatthe AAexp of the experimental signal (used in the denominator ofEq. (1)) is 37% smaller when the edge effect is not considered, whatwould increase the value of accuracy measure AA for 59% in thisexample. This means that all integral variables (integrated break



6 A. Prosek et al. / Nuclear Engineering and Design xxx (2008) xxx–xxx

Table 2Calculation of AA and AAm for LOFT L2-5 pressurizer pressure in time interval(0–20 s)

ID AAerr AA AAerr m AAm AAerr/AAerr m Rank AA Rank AAm

CEA 4.28 0.136 3.63 0.209 1.2 11 9GID 0.86 0.027 0.88 0.050 1.0 1 1GRS 1.34 0.043 1.14 0.066 1.2 2 2IRSN 4.13 0.131 3.64 0.209 1.1 10 10JNES 6.80 0.216 4.76 0.274 1.4 13 13KAERI 2.91 0.092 2.40 0.138 1.2 7 7KFKI 1.80 0.057 1.35 0.078 1.3 5 4KINS 2.06 0.065 1.67 0.096 1.2 6 5NRI-1 1.73 0.055 2.16 0.124 0.8 4 6NRI-2 1.70 0.054 1.24 0.071 1.4 3 3PSI 4.01 0.128 3.86 0.222 1.0 9 11TAEK 3.73 0.118 2.55 0.147 1.5 8 8UPC 9.86 0.314 6.92 0.399 1.4 14 14UPI 5.77 0.183 4.34 0.250 1.3 12 12

ID AAexp AAexp m AAexp/AAexp m

EXP 31.45 17.37 1.8

flow, ECCS injected mass) and variables dropping from nominal tolow value (power, primary pressure during LOCA) exhibit lower AAvalues just because the artificially introduced unphysical edge ispresent in the experimental signal. This basically explains the, ingeneral, very high accuracy of these integral variables (Prosek et al.,2002) comparing to other variables and why the acceptability factorfor primary pressure (D’Auria et al., 1994) (dropping during smallbreak LOCAs) had to be set to the low value 0.1 (for other param-eters there is no need for a special criterion, so the total accuracycriterion 0.4 is used).

The improved FFTBM by signal mirroring provides a realistic,unbiased and consistent judgment, since it eliminates the effect ofthe unphysical edge, which sometimes is present and sometimesnot. For example, when comparing primary pressures, during blow-down the pressure is decreasing and so a huge edge is present(it significantly decreases AA calculated by the original FFTBM),while during a very small break the pressure may recover to normalpressure after the initial drop due to emergency core cooling injec-tion and consequently there is no edge (the original FFTBM thencalculates similar values of AA as the improved FFTBM by signalmirroring).

3.3. Case 2 study by signal mirroring


To further demonstrate how signal mirroring works, in the sec-ond example the pressurizer pressure accuracy of LOFT L-2 testcalculations is calculated for two time intervals, the blowdownphase time interval (0–20 s) and the whole transient time inter-val (0–119.5 s), as shown in Tables 2 and 3, respectively. Both, theoriginal FFTBM and the improved FFTBM by signal mirroring wereused. For each calculation the values of the average amplitude ofthe error signal (see Eq. (4)) and the average amplitude per Eq.(1) are shown with the corresponding average amplitude of theexperimental signal (see Eq. (5)). It should be noted that two cal-culations (GRS, KAERI) did not provide data for the whole transienttime interval, therefore for them the quantitative assessment wasnot applicable.

To see the influence of the edge elimination, the ratios ofaverage amplitudes of the error signal obtained by the originalFFTBM and the improved FFTBM by signal mirroring are shownin Tables 2 and 3. Besides the ratios, average amplitudes of theerror signal, average amplitudes of the experimental signal, aver-age amplitudes and rank of average amplitudes for both, the originalFFTBM and the improved FFTBM by signal mirroring are shown. It

Table 3Calculation of AA and AAm for LOFT L2-5 pressurizer pressure in time interval(0–119.5 s)

ID AAerr AA AAerr m AAm AAerr/AAerr m Rank AA Rank AAm

CEA 3.06 0.096 3.85 0.237 0.8 8 8GID 1.08 0.034 1.23 0.076 0.9 1 1GRS N.A. N.A. N.A. N.A. N.A. N.A. N.A.IRSN 3.10 0.097 3.97 0.244 0.8 9 9JNES 3.93 0.123 4.96 0.305 0.8 11 11KAERI N.A. N.A. N.A. N.A. N.A. N.A. N.A.KFKI 1.17 0.036 1.38 0.085 0.8 3 3KINS 1.49 0.047 1.80 0.111 0.8 4 4NRI-1 1.64 0.051 2.09 0.129 0.8 5 5NRI-2 1.10 0.034 1.34 0.082 0.8 2 2PSI 2.97 0.093 3.74 0.230 0.8 7 7TAEK 2.23 0.070 2.71 0.167 0.8 6 6UPC 5.96 0.186 7.44 0.458 0.8 12 12UPI 3.80 0.119 4.47 0.275 0.8 10 10

ID AAexp AAexp m AAexp/AAexp m

EXP 32.00 16.26 2.0

can be seen that the average amplitude of the experimental signalis similar for both time intervals. The reason is that after 20 s thepressure signal (see Fig. 3(a) left) is not changing very much. Asthe pressure at 20 s significantly dropped, the edge effect at 20 s israther similar to the edge effect at 119.5 s. The average amplitudes ofthe experimental signal obtained by the original FFTBM are 1.8 and2.0 times larger than by the improved FFTBM by signal mirroringfor the first and second time interval, respectively. The conclusionfor the error signals (see Fig. 3(b) left) is different. The ratio of theaverage amplitudes of the error signal varies between 0.8 and 1.5 inthe first time interval, while in the second time interval this ratio isaround 0.8. The reason for the varying ratio in the first time intervalis that the edges between calculations are quite different, while inthe second time interval the edges are rather similar between calcu-lations. Ranking of the AA values may change only in the case whenthe ratio of AAerr varies, i.e. in the first time interval, as it can be seenfrom Table 2. In the second time interval (in the whole calculation)the rank of AA remains unchanged, as shown in Table 3. Neverthe-less, absolutely the AA changes when the edge is not considered inthe experimental signal and this influences the total accuracy.

3.4. Application to LOFT L2-5 test


The quantitative assessment of code calculations was performedfor a sequence of time intervals incremented for 5 s as we progressinto the transient. In this way time dependent code accuracy wasobtained. For the analysis the set of 18 time trends shown inTable 4 were used as proposed in the frame of the BEMUSE project(OECD/NEA, 2006).

The results of the accuracy analysis are shown in Fig. 3, on leftfor the pressurizer pressure (P4) and on the right for the rod surfacetemperature (P16). In total 14 different calculations were com-pared. For each variable the comparison between experimental andcalculated data is shown (Fig. 3(a)), the error signal between the cal-culation and the experiment (Fig. 3(b)), the AA calculated with theoriginal FFTBM (Fig. 3(c)), and AAm calculated with the improvedFFTBM by signal mirroring (Fig. 3(d)). For the pressurizer pressuresignal (Fig. 3 (left)) the comparison of trends and error signals showthat during the first 30 s the pressurizer pressure was in most casesunderpredicted. Therefore, the accuracy should decrease with timeduring this time interval. In the case of the original FFTBM the shapeof the calculated AA is similar to the absolute value of the error sig-nal because in the calculation of AAerr the edge effect dominates.It should be noted that the value of the edge is equal to the error



A. Prosek et al. / Nuclear Engineering and Design xxx (2008) xxx–xxx 7


Fig. 3. LOFT L2-5 test trends of pressurizer pressure (left) and rod surface temperature in

signal at that discrete point when at time 0 the error is zero. Theedge effect impacts also AAexp, which increases till 30 s. The AAexp

and AAerr influence the AA. In the case of the improved FFTBM bysignal mirroring the value of AAm increases (accuracy decreases)as long as the error signal is present. After 30 s the AAm stabilizes.However, the value is higher than in the case of AA because the edgeeffect is eliminated from AAexp.

For the rod surface temperature shown in Fig. 3 (right) the cal-culated maximum values of the rod surface temperature were in


hot rod—2/3 core height (right): (a) trends; (b) error signals; (c) AA; (d) AAm.

rather good agreement with the experimental value. However, thetrends were in general under or over predicted, with some calcula-tions that predicted quench too early. From the error signal shownin Fig. 3(b) it can be seen that discrepancies were present until corequench. From Fig. 3(c) it can be seen that the edge effect is presentin the calculation of AA. When looking AAm in Fig. 3(d) it can beseen that the value of AAm monotonically increases as long as thediscrepancy is present. Nevertheless, when considering the wholetransient time interval, there is only a slight difference between AA


ARTICLE ING ModelNED-5071; No. of Pages 10


Table 4Variables used in quantitative analysis of LOFT L2-5 test

ID Description of variable

and P11, which are monotonic functions, are judged significantly

P1 Intact loop pressure in hot legP2 SG pressure—secondary sideP3 Accumulator A pressureP4 Pressurizer pressureP5 Core inlet fluid temperatureP6 Core outlet fluid temperatureP7 Upper head fluid temperatureP8 SG downcomer bottom fluid temperatureP9 Break flow rate in cold legP10 Break flow rate in hot legP11 ECCS integral break flow rateP12 Primary side total massP13 SG pressure drop—U tube primary sideP14 Primary pumps pressure dropP16 Rod surface temperature in hot rod in hot channel (zone 4)—2/3

core heightP18 Rod surface temperature in average rod in average channel (zone

2)—bottom levelP19 Rod surface temperature in average rod in average channel (zone

2)—2/3 core heightP20 Rod surface temperature in average rod in average channel (zone

2)—top level

and AAm. The reason is the small edge in the whole transient timeinterval. This is the reason why the original FFTBM in several cases


produced reasonable results (but not for monotonically increasingand decreasing signals like the pressurizer pressure). Nevertheless,for investigating the influence of discrepancies as we progress intothe transient the edge effect needs to be eliminated to make theright conclusions. Only the improved FFTBM by signal mirroringgives consistent results. Consistent judgment of the time depen-dent accuracy is very important as the analyst in this way gets anobjective picture how each discrepancy decreases the accuracy. Onthe other hand, from Fig. 3(c) it can be very easily verified thatthe requirements for accuracy measures (Ambrosini et al., 1990)that at any time into the transient the previous history should beremembered and that the measure should be independent upon thetransient duration are not fulfilled in the case of the original FFTBMwhen the edge influences the results. Through performing the timedependent accuracy, tens of calculations for different time intervalswere performed demonstrating the consistency of the improvedFFTBM by signal mirroring comparing to the original FFTBM andtherefore there is no need to use further experiments for the vali-dation of the improved FFTBM. On the other hand, very frequentlyat the end of the transient the edges are rather small and in such

Table 5Correction factors for accuracy judgment with original FFTBM for LOFT L2-5 test

Variable/calc. P1 P2 P3 P4 P5 P6 P7 P8 P

CEA 1.4 0.9 1.8 2.1 1.2 1.3 0.9 0.9 0GID 1.5 1.1 1.2 1.9 1.3 1.2 1.3 0.8 0GRSa 1.5 0.9 1.8 2.0 1.3 1.3 0.8 0.8 0IRSN 1.5 1.0 1.8 2.1 1.3 1.3 1.2 0.9 0JNES 1.6 0.5 1.8 2.1 1.1 1.3 1.2 0.6 0KAERIb 1.6 1.0 1.6 2.1 1.1 1.3 1.2 0.6 0KFKI 1.6 0.8 1.9 1.9 1.2 1.3 1.3 1.1 0KINS 1.5 1.0 1.6 2.0 1.2 1.2 1.2 0.8 0NRI-1 1.6 1.1 1.2 2.1 1.3 1.3 1.2 0.8 0NRI-2 1.4 0.9 1.9 2.0 1.3 1.3 1.2 1.0 0PSI 1.5 0.9 2.0 2.1 1.3 1.2 1.2 0.8 0TAEK 1.5 0.9 1.9 2.0 0.9 1.2 1.2 0.7 0UPC 1.5 0.9 1.2 2.0 1.2 1.3 1.2 0.9 0UPI 1.5 1.1 1.7 1.9 1.2 1.3 1.2 0.8 0Average 1.5 0.9 1.7 2.0 1.2 1.3 1.2 0.8 0

a For time interval (0–110 s).b For time interval (0–100 s).

PRESSand Design xxx (2008) xxx–xxx

cases also the original FFTBM produces consistent results. Luckily,this was the case in several studies performed with the originalFFTBM. Nevertheless, before the analyst is confident to the resultsobtained by the original FFTBM he should always verify that theedge is not present in the signal. In the opposite, the results aredoubtful. It should be also noted that the methodology using origi-nal FFTBM requires qualitative analysis with visual observation andonly for discrepancies which are reasonable and understood thequantitative assessment using FFTBM could be done.

Finally, in Table 5 the correction factors for the LOFT L2-5 calcu-lation in the whole time interval derived from quantitative resultsobtained by the original FFTBM and the improved FFTBM by signalmirroring are shown. For time trends, the reader can refer to OECD,2006. The correction factors tell us how much the accuracy obtainedby the original FFTBM would need to be corrected because of theedge contribution to the results. These factors were obtained as theratio between the normalized AAm and AA. The AA and AAm werenormalized by all calculations and all selected variables for quan-titative assessment (average values of normalized AA and AAm are1). In this way the characteristics of the original and the improvedFFTBM were captured. When the ratio is 1, the edge contributionto the original FFTBM is equal to the average edge contribution inthe calculations used for normalization. When the ratio is greaterthan 1 this means that the original FFTBM judges the variable toogood due to the edge effect. For example, the variables P1, P3, P4


too good. Also temperatures, which are decreasing with progress-ing into the transient, are judged a little bit too good due to theedge influence (see P5, P6, P7, P18, P19, and P20). It should be notedthat also P16 has an edge in the experimental signal but this influ-ence is relatively smaller due to the more complex shape of theexperimental signal, which has more harmonic components in thefrequency domain than the other experimental signals. Therefore,the edge contribution from the error signal prevails the edge contri-bution from the experimental signal what results in a ratio slightlysmaller or equal to 1. A ratio smaller than 1 means that the originalFFTBM judges the variable too bad due to the edge effect. For exam-ple, the variables P9, P10, P13 and P14 have an edge contributiondue to the still present discrepancy at the end of the transient andthe edge contribution in the error signal prevails the rather smalledge contribution in the experimental signal. Table 5 shows thatthe assessment by the original FFTBM qualitatively agrees (±20%)with the assessment with the improved FFTBM by signal mirror-ing only for around 2/3 of variables. For monotonic variables thedisagreement is much larger. Finally, in the case of the rod surfacetemperature (P16) the agreement is so good that practically the

9 P10 P11 P12 P13 P14 P16 P18 P19 P20

.9 0.9 1.4 1.0 0.9 0.8 0.9 1.1 1.2 1.2

.8 0.9 1.0 0.9 0.9 0.8 1.0 1.1 1.2 1.2

.8 0.9 1.1 0.7 0.8 0.8 0.9 1.2 1.2 1.2

.9 0.9 1.4 1.0 0.9 0.8 0.9 1.1 1.2 1.2

.9 0.9 1.5 1.0 0.9 0.8 0.9 1.1 1.2 1.2

.8 0.9 1.1 1.0 0.9 0.8 0.9 1.1 1.2 1.1

.8 0.9 1.1 0.9 0.9 0.8 1.0 1.1 1.2 1.2

.8 0.9 1.2 1.0 0.9 0.8 0.9 1.1 1.2 1.2

.8 0.8 1.7 1.2 0.9 0.8 0.9 1.1 1.2 1.2

.8 0.9 1.6 1.2 0.9 0.8 0.9 1.1 1.1 1.2

.8 0.9 2.0 0.9 0.9 0.8 1.0 1.1 1.2 1.2

.9 0.9 1.3 1.3 0.9 0.8 1.0 1.1 1.1 1.2

.9 0.8 1.5 1.4 0.8 0.8 1.0 1.1 1.2 1.2

.9 0.9 1.6 1.3 0.9 0.8 0.9 1.1 1.1 1.2

.8 0.9 1.4 1.1 0.9 0.8 0.9 1.1 1.2 1.2


INering


A. Prosek et al. / Nuclear Engine

same accuracy conclusion could be derived. This is due to the fact,that at the end of the transient the edge impact is practically negligi-ble. Please note that the time dependent accuracy for P16 obtainedwith the original FFTBM, shown in Fig. 3 (right), is incorrect duringthe period from 40 to 60 s when the edge effect is large.

3.5. Discussion of results

A demonstration application of the improved FFTBM by signalmirroring was done for a design basis accident. In the case of theLOFT L2-5 test calculation it was shown that only the improvedFFTBM by signal mirroring gives a realistic judgment for the timedependent accuracy. The differences between AA and AAm as afunction of time were clearly shown to be due to the edge con-tribution. On the other hand, for the whole transient time intervalwith stabilized conditions resulting in small edges also the judg-ment by the original FFTBM is qualitatively correct. However, thisis never the case for monotonic trends where the edge increaseswith increasing the transient time. In the case of the LOFT L2-5test at 107 s the LPIS injection was terminated and after the testwas considered to be completed. At this time approximately 2/3of variables were at stabilized conditions with rather small edges.However, in general there is a need to make comparisons for anytime window and the transient may not be terminated at stableconditions resulting in small edges.

For the proposed improved FFTBM by signal mirroring theacceptability criteria need to be defined in the same way as thiswas done for the original FFTBM. The easiest way would be to usethe same set of calculations as for the original FFTBM. The obtainedresults for LOFT L2-5 suggest slightly higher acceptability limitsfor the improved FFTBM by signal mirroring than for the originalFFTBM. In Table 5 it was shown how to correct AA and this gives animpression how much the acceptability limits should be changed.It seems that the restrictive pressure criterion in the original FFTBMwas set based on pressure trends during small break LOCAs infacilities simulating typical PWRs (high initial pressure and largepressure drop, therefore high edge). When tests on different facili-ties were simulated, there were difficulties in satisfying the primarypressure criterion. The first example is the accuracy quantifica-tion of four standard problem exercises (SPEs) organized by IAEA(D’Auria et al., 1996). In this study, only the primary system pressurehas been considered. Among other things it was also concluded thatin the case of SPE-3 the calculation is clearly unacceptable (AA was0.31) and that more complex transients lead to worse results than


simple one’s. As no plots are shown in the paper by D’Auria et al.(1996) no further conclusion can be done except that the pressuredrop (edge) is smaller than in a typical PWR. Namely, the initialpressure in this test is lower than in the typical PWR test. By low-ering the pressure edge the values of AA are increased. This can bestill better illustrated in the recent application of FFTBM to heavywater reactors. In the study (Prosek et al., 2006) all participants ful-filled the acceptance criterion for the total accuracy K < 0.4 while theprimary pressure criterion was not fulfilled. In the blind accuracycalculation the AA value for the primary pressure was 0.117 for thebest calculation. The header 7 pressure with initial pressure around10 MPa was selected as a variable representing the primary pres-sure. In the open accuracy analysis header 6 pressure was proposedby a representative from Italy. The initial value of this pressure wasaround 12 MPa. Now the value of AA was below 0.1 for most of par-ticipants mostly due to the increased pressure edge effect (the bestAA was 0.074) due to the higher pressure.

The last example of the AA calculation for the primary pressureis for ISP-22 calculations (loss of feedwater test). From paper byProsek et al. (2002) it may be seen that the AA value for the pri-mary pressure in the best posttest calculation is 0.21, the worst (as

PRESSand Design xxx (2008) xxx–xxx 9

judged by the original FFTBM) among summarized ISPs. From theoriginal report (Ambrosini et al., 1992) showing plots it can be easilyconcluded that the edge contribution in the experimental signal issmaller than typically for small break LOCAs due to lower pressuredrop and the complex shape of the experimental signal, resultingin larger AA.

All these examples demonstrate that due to the unpredictableedge contribution a consistent criterion for the primary pressurecannot be defined for the original FFTBM, while for the improvedFFTBM with signal mirroring this can be done.

4. Conclusions

In the past the most widely used method for code accuracyquantification of primary system thermal-hydraulic codes was theoriginal FFTBM. Recently, in the original FFTBM an important defi-ciency was discovered. It turned out that the accuracy measuredepends on the difference between the first and last point of theinvestigated signal. Namely, the DFT mathematical method, onwhich the FFTBM is based, treats the investigated finite length sig-nal as an infinite length periodic signal, introducing discontinuitiesif the first and last data point of the finite signal differ. These dis-continuities produce a variegated spectrum of frequencies whenapplying DFT, which may overshadow the frequency spectrum ofthe investigated signal. This so called edge effect is a significant defi-ciency of the original FFTBM since for the comparison the shape ofthe investigated signal is important and not the artificially intro-duced unphysical edge.

Therefore, we proposed to resolve the edge effect problem ona unique way by signal mirroring, where the investigated signal ismirrored before the FFTBM is applied. By composing the originalsignal and its mirrored signal a symmetric signal with the samecharacteristics is obtained, but without introducing artificial dis-continuities when viewed as a periodically extended infinite signal.With the so improved FFTBM by signal mirroring a consistent andunbiased tool for quantitative assessment is obtained. An additionalgood property of the improved FFTBM is that the same FFTBM pro-cedure (numerical tools, etc.) may be applied as with the originalFFTBM.

The benefits of the improved FFTBM by signal mirroring weredemonstrated on the large break LOCA test LOFT L2-5. The resultsshow that the so improved FFTBM judges the accuracy of variablesin a reliable, unbiased and consistent way. For the LOFT L2-test itwas shown that the results for the whole transient time interval,


obtained by the original FFTBM and the improved FFTBM by signalmirroring, qualitatively agree for variables with a small edge anddiffer for variables with a large edge, as was expected. The results forthe time dependent accuracy with the improved FFTBM by signalmirroring showed, that the analyst can now get the true informa-tion how the discrepancies influence the accuracy. Based on thisone can conclude that the improved FFTBM by signal mirroringrealistically judges the accuracy and is general applicable.

Acknowledgments

The authors acknowledge the support of the Ministry of highereducation, science and technology of the Republic of Sloveniawithin the program P2-0026 and the research projects J2-6542 andJ2-6565. Special thanks also to the University of Pisa, with whichwe cooperate in the BEMUSE Phase II, for sending the data of theLOFT L2-5 experiment and the calculation results.

References

Ambrosini, W., Bovalini, R., D’Auria, F., 1990. Evaluation of accuracy of thermalhy-draulic code calculations. Energia Nucleare 7, 5–16.


IN


Ambrosini, W., Breghi, M.P., D’Auria, F., Galassi, G.M., 1992. Evaluation of post-testanalyses of OECD-CSNI International Standard Problem 22. Report, University ofPisa, NT 184 (91) Rev. 1.

Bang, Y.S., Seul, K.W., Oh, D.Y., Kim, I.-G., Kim, B.H., Kim, H.K., Kim, H.J., 2004.An uncertainty evaluation method and application to prediction of LOFT L2-5experiment. In: Proceedings—International Meeting on Updates in Best EstimateMethods in Nuclear Installation Safety Analysis, BE-2004, pp. 324–330.

D’Auria, F., Giannotti, W., 2000. Development of a code with the capability of internalassessment of uncertainty. Nucl. Technol. 131, 159–196.

D’Auria, F., Ingegneri, M., 1997. Qualitative and quantitative analysis of CATHARE2code results of counterpart test calculations in LOBI, SPES, BETHSY, LSTF facilities.In: Proceedings of NURETH-8, vol. 2, pp. 1195–1203.

D’Auria, F., Leonardi, M., Pochard, R., 1994. Methodology for the evaluation of ther-malhydraulic codes accuracy. In: Proceedings of the International Conference onNew trends in Nuclear System Thermohydraulics, Pisa, Italy, pp. 467–477.

D’Auria, F., Eramo, A., Frogheri, M., Galassi, G.M., 1996. Accuracy quantification inSPE-1 to SPE-4 organised by IAEA. In: Proceedings of the International Con-ference on Nuclear Engineering (ICONE-4), vol. 3, New Orleans, Louisiana, pp.461–469.

Kunz, R.F., Kasmala, G.F., Mahaffy, J.H., Murray, C.J., 2002. On the automated assess-ment of nuclear reactor systems code accuracy. Nucl. Eng. Des. 211, 245–272.

Leskovar, M., Mavko, B., 2006. Simulation of the Phebus FPT1 severe accident exper-iment with the MELCOR computer code. J. Mech. Eng. 52 (3), 142–160.

Mavko, B., Prosek, A., D’Auria, F., 1997. Determination of code accuracy in predictingsmall-break LOCA experiment. Nucl. Technol. 120, 1–19.

PRESS


and Design xxx (2008) xxx–xxx

OECD/NEA, 2006. BEMUSE Phase 2 Report: Re-Analysis of the ISP-13 Exercise, PostTest Analysis of the LOFT L2-5 Test Calculation, OECD/NEA Report, Committeeon the Safety of Nuclear Installations (CSNI), NEA/CSNI/R (2006)2.

Prosek, A., Leskovar, M., 2005. Application of FFTBM to Severe Accidents. In: Pro-ceedings of the International Conference on Nuclear Energy for New Europe,Bled, Slovenia, pp. 013.1–013.10.

Prosek, A., Leskovar, M., 2006. Improved FFTBM by signal mirroring. IJS Report, JozefStefan Institute, IJS-DP-9336.

Prosek, A., Mavko, B., 2003. A tool for quantitative assessment of code calculationswith an improved fast Fourier transform based method. Electrotech. Rev. 70 (5),291–296, http://ev.fe.uni-lj.si/5-2003/prosek.pdf.

Prosek, A., Mavko, B., 2004. Quantitative assessment of time trends: influenceof time window selection. In: Proceedings of the 5th International Confer-ence on Nuclear Option in Countries with Small and Medium Electricity Grids,Dubrovnik, Croatia, pp. 1–9.

Prosek, A., D’Auria, F., Mavko, B., 2002. Review of quantitative accuracy assessmentswith fast Fourier transform based method (FFTBM). Nucl. Eng. Des. 217 (1/2),179–206.

Prosek, A., D’Auria, F., Richards, D.J., Mavko, B., 2006. Quantitative assessment ofthermal–hydraulic codes used for heavy water reactor calculations. Nucl. Eng.Des. 236, 295–308.

Smith, S.W., 1999. The Scientist and Engineer’s Guide to Digital Signal Processing,2nd ed. California Technical Publishing, San Diego, California.


http://ev.fe.uni-lj.si/5-2003/prosek.pdf

Quantitative assessment with improved fast Fourier transform based method by signal mirroring

Documents