Statistical Analysis of CFD Solutions from the 3rd AIAA ... · Statistical Analysis of CFD Solutions from the 3rd AIAA Drag Prediction ... Configuration Aerodynamics Branch NASA Langley

1

DPW-3DPW-3Statistical Analysis of CFD Solutions

from the 3rd AIAA Drag PredictionWorkshop

J. H. MorrisonComputational AeroSciences Branch

M. J. HemschConfiguration Aerodynamics Branch

NASA Langley Research Center

3rd AIAA APA Drag Prediction WorkshopJune 3-4, 2006

2

DPW-3DPW-3Outline of the talk

• Method for analyzing the collective

• Case 1 DLR-F6 and FX2B Fairing– Individual Solution Analysis– Grid Convergence Study– Comparison with DPW-2

• Case 2 Grid Convergence Study for DPW-W1

• Summary

• Concluding Remarks

3

DPW-3DPW-3Analysis Method

• Grid Convergence for nested solutions– Reduction in spread?– Reduction in scatter of “core” solutions?– Significant changes in medians?– Compare DPW-2 and DPW-3 spread and

scatter

4

DPW-3DPW-3

The median gives a robust estimateof the population mean.

Analysis Method (2)

5

DPW-3DPW-3

Case 1: DLR-F6 Wing Body andDLR-F6 with FX2B Fairing

6

DPW-3DPW-3Case 1 Solution Statistics

11121215Codes

10101114Institutions

11121215Authors

14161316Solutions

CoreNestedCoreNested

DPW-3DPW-2

7

DPW-3DPW-3Solution Analysis

8

DPW-3DPW-3Solution Analysis (2)

CD_PR

CD_SF

9

DPW-3DPW-3Solution Analysis (3)

ALPHA

CM_TOT

10

DPW-3DPW-3Solution Analysis (4)

Same CodeSame GridDifferent Turbulence Models

Same Grid Nodes; Different CellsSame Turbulence ModelDifferent Code

11

DPW-3DPW-3• Scatter in solutions is due to:

– Numerical error– Modeling error (e.g. physics models, computational models)– User errors– Code errors

• For grid convergence, numerical error asymptoticallyapproaches zero leaving the other three contributors

• For the collective to show convergence, the followingwould have to happen:– The ranges for the configurations would approach a constant as

the grid “improved”.– The scatter (standard deviation) of the “core” solutions would

approach a constant as the grid improved.– The medians of the core solutions would change asymptotically.

What does convergence look like?

12

DPW-3DPW-3Grid Sizes

C M F

13

DPW-3DPW-3Nested CD_TOTF6 FX2B

14

DPW-3DPW-3Nested CD_TOT minus Outliers

F6 FX2B

15

DPW-3DPW-3Convergence of CD_TOT

C M FNPTS-2/3

F

M

C

16

DPW-3DPW-3Convergence CD_PR, CD_SF

C

M

F

NPTS-2/3

F M C

NPTS-2/3

CD_PR CD_SF

17

DPW-3DPW-3Convergence of ALPHA & CM_TOTALPHA

C M F

CM_TOT

C M F

18

DPW-3DPW-3Convergence of SpreadCD_TOT

C M F

19

DPW-3DPW-3Convergence of Spread (2)CD_PR

C M F

CD_SF

C M F

20

DPW-3DPW-3Convergence of Spread (3)ALPHA

C M F

CM_TOT

C M F

21

DPW-3DPW-3Convergence of Core IntervalCD_TOT

C M F

22

DPW-3DPW-3Convergence of Core Interval (2)CD_PR

C M F

CD_SF

C M F

23

DPW-3DPW-3Convergence of Core Interval (3)ALPHA

C M F

CM_TOT

C M F

24

DPW-3DPW-3

Case 2: DPW-W1 Wing Alone

N.B. DPW-W2 has not been analyzed

25

DPW-3DPW-3Case 2 Solution Statistics

66Codes

66Institutions

66Authors

77Solutions

CoreNested

DPW-W1 Wing Alone

26

DPW-3DPW-3W1 Nested CD_TOT

27

DPW-3DPW-3Convergence of CD_TOT, CL_TOT

NPTS-2/3

CL_TOT

NPTS-2/3

CD_TOT

28

DPW-3DPW-3Convergence of CM_TOT

NPTS-2/3

CM_TOT

29

DPW-3DPW-3Spread of CD_TOT, CL_TOT

NPTS-2/3NPTS-2/3

Spread of CD_TOT Spread of CL_TOT

30

DPW-3DPW-3CD_TOT, CL_TOT Core Interval

NPTS-2/3NPTS-2/3

CD_TOT Core Interval CL_TOT Core Interval

31

DPW-3DPW-3

Summary

32

DPW-3DPW-3Spread (of nested data) for Finest Grid

ALPHA CD_TOT CD_PR CD_SF CM_TOT

33

DPW-3DPW-3Core Interval for Finest Grid

ALPHA CD_TOT CD_PR CD_SF CM_TOT

34

DPW-3DPW-3Concluding Remarks• The Good News:

– DPW-3 was a “blind test”, i.e. no experimental data existed to“guide” solutions. The results were about as good for the blindtest as for DPW-2.

– DPW-W1 might be showing evidence that it is in the asymptoticrange

• The Less Good News:– Have not demonstrated convergence of medians, spread or core

interval for F6/FX2B despite increased grid sizes– F6 spread and core interval have not improved from DPW-2– FX2B spread and core interval are not substantially better than F6– DPW-W1 spread and core interval are not showing convergence– After 3 drag prediction workshops, grids remain a leading order issue

35

DPW-3DPW-3Concluding Remarks (2)

Hemsch’s remarks from DPW-2 still apply:

• Regarding grid convergence for the collective:– There is no reduction in spread;– There is no reduction in core scatter;– The medians MAY be converging, although it

can’t be proven with the present results.

36

DPW-3DPW-3Some Recommendations

• We must make a concerted effort to understandthe differences in the codes and models

• We must make a concerted effort to understandthe effects of grid quality and grid resolution

• We must analyze and improve our processes

37

DPW-3DPW-3

Fini?

38

DPW-3DPW-3Grid Convergence – All Solutions