Two stage group screening with noise factors, unequal group sizes and differing probabilities of active effects Anna Vine University of Southampton, UK.

Two stage group screening with noise

factors, unequal group sizes and differing

probabilities of active effects

Anna VineUniversity of

Southampton, UK

Susan Lewis, Angela Dean

Funded by EPSRC

Summary

• Two-stage group screening

– for control and noise factors

– Interaction group screening

• Criteria for comparing strategies

• Example

– Software to guide experimenters

Screening in industrial experiments

• Response can depend on a large number of different factors of two types:

– control: can be set by engineers during manufacturing

– noise: can’t be controlled in manufacturing/use but can be controlled in experiments

• Aim of screening experiments is to identify important factors

– judged by engineers to give a substantial improvement, Δ > 0, in product performance

Two-stage group screening

• Individual two-level factors divided into groups:– F groups of control factors with sizes

– N groups of noise factors with sizes

• Define a grouped factor for each group by setting all factors within a group to their high (low) level

• Two stages of experimentation– Stage I on grouped factors– Stage II on factors in groups found to be

important at stage I

)()(1 ,..., c

Fc gg

)()(1 ,..., n

Nn gg

Active factors• In interaction group screening main effects and

interactions of the grouped factors are estimated at stage I

• A grouped control factor is declared active

– if it has a detected main effect

– if it is in a detected control x control interaction

– if it is in a detected control x noise interaction

• A grouped noise factor is only declared active if in a detected interaction with a control factor

• Detection of active grouped factors is not certain

Probabilities of effects being active

• The total number of effects S that require estimation over the two-stage experiment is a random variable

• Based on experts’ opinions assign prior probabilities

– to each individual main effect being active

– to each individual interaction being active

– or use heredity principle (Chipman 1996)

• Calculate probabilities of each grouped effect being active from individual probabilities

Criteria for choice of groupings

Aim to

1. minimise expected total number of effects that require estimation, E(S)

2. minimise the probability of exceeding a target number u of factorial effects requiring estimation, P(S > u)

3. minimise the risk of failing to detect important main effects and interactions

• Conflict between aims

• Lewis and Dean (2001) and Dean and Lewis (2002)

Formulating the criteria

• Expected total number of individual effects requiring estimation

∑∑1= 1=

++2

1++2=

F

i

N

j

cnji

nj

ci pggFN

FNSE )(

,)()()(

∑ ∑∑ ∏1

1= 1+=1= 1=

+11+F

i

F

ik

ccki

ck

ci

N

j

F

i

cnji

nj pggpg )(

,)()()(

,)( )](][[

∑ ∏ ∏1= 1= ≠1=

111121++F

i

N

j

F

ikk

ci

ccki

cnji

ci

ci pppgg

,

)()(,

)(,

)()( )])(()(][/)([

∏∏ ∏ ∏∏1= 1= ≠1= 1= 1=

+111F

i

N

j

F

ikk

F

i

N

j

cnji

ci

ccki

cnji pippp

,

)(,

)()(,

)(, )()])()([(

• Probability distribution of the total number of individual effects requiring estimation

∑321

==R

cnt

cct

ct IIIPsSP ),,()( )()()(

Practical situation

• Partition individual control factors into two types: very likely and less likely

• Control factors believed very likely to be active are assigned the same high probability of a main effect being active

• Assign very likely and less likely control factors to separate groups

Example

To illustrate use of criteria consider:

• 7 individual control factors - main effects probabilities 1

• 8 individual control factors- main effects probabilities 0.2

• 4 individual noise factors- main effects probabilities 0.3

• individual control x noise interaction probabilities 0.07

• individual control x control interaction probabilities 0.05

Distribution of S

Risk of exceeding a target

Software

• Interactive web based software

– allows elicitation of information from experts

– enables comparison of different groupings under criteria 1 and 2

– allows simulations of group screening experiments to be run for criterion 3

Use of methods/software

• Planned a group screening experiment on engine cold start optimisation at Jaguar Cars

Investigation of groupings

Very likely control

Less likely

control

Noise

E(S)sd(S

)P(S>120

)P(S>150

)P(S>180

)Grouping

7 8 2,2 174.5 20.31

0.98 0.89 0.55

7 2,6 2,2 158.41

21.97

0.94 0.70 0.16

7 3,5 2,2 155.32

23.13

0.92 0.61 0.13

7 4,4 2,2 154.43

23.56

0.91 0.60 0.12

7 2,2,4 2,2 143.16

21.46

0.85 0.38 0.03

7 2,3,3 2,2 141.14

21.61

0.83 0.35 0.02

7 2,2,2,2 2,2 134.04

19.51

0.76 0.20 0.01

2,5 8 2,2 159.22

21.71

0.94 0.68 0.13

2,5 2,6 2,2 142.42

21.19

0.84 0.37 0.02

2,5 3,5 2,2 138.49

21.45

0.79 0.29 0.01

2,5 4,4 2,2 137.28

21.57

0.78 0.27 0.01

2,5 2,2,4 2,2 127.99

18.97

0.67 0.12 0.00

2,5 2,3,3 2,2 125.79

18.76

0.62 0.09 0.00

2,5 2,2,2,2 2,2 120.85

16.42

0.52 0.04 0.00

3,4 8 2,2 156.64

22.60

0.93 0.63 0.15

3,4 2,6 2,2 139.17

21.34

0.80 0.30 0.02

3,4 3,5 2,2 134.94

21.27

0.75 0.24 0.01

3,4 4,4 2,2 133.63

21.27

0.73 0.21 0.01

3,4 2,2,4 2,2 124.45

18.56

0.60 0.08 0.00

3,4 2,3,3 2,2 122.18

18.23

0.54 0.06 0.00

3,4 2,2,2,2 2,2 117.41

15.84

0.43 0.02 0.00

2,2,3 8 2,2 146.15

20.86

0.88 0.43 0.04

2,2,3 2,6 2,2 130.39

18.86

0.71 0.15 0.00

2,2,3 3,5 2,2 126.19

18.55

0.63 0.09 0.01

2,2,3 4,4 2,2 124.89

18.44

0.61 0.08 0.00

2,2,3 2,2,4 2,2 117.85

15.72

0.44 0.02 0.00

2,2,3 2,3,3 2,2 115.69

15.34

0.38 0.01 0.00

2,2,3 2,2,2,2 2,2 112.97

13.00

0.27 0.00 0.00

Investigation of groupings - CGS

Very likely control

Less likely

control

Noise

E(S)sd(S

)P(S>80)

P(S>110)

P(S>140)

Grouping

7 8 2,2 141.75

46.16

0.83 0.83 0.63

7 2,6 2,2 118.89

42.89

0.76 0.62 0.32

7 3,5 2,2 114.16

43.21

0.79 0.59 0.25

7 4,4 2,2 112.68

43.46

0.72 0.47 0.26

7 2,2,4 2,2 99.95 36.45

0.62 0.36 0.13

7 2,3,3 2,2 97.77 35.80

0.69 0.34 0.15

* 7 2,2,2,2 2,2 88.07 29.63

0.48 0.19 0.04

2,2,3 8 2,2 143.75

46.16

0.83 0383 0.63

2,2,3 2,6 2,2 120.89

42.89

0.76 0.62 0.32

2,2,3 3,5 2,2 116.16

43.21

0.79 0.63 0.34

2,2,3 4,4 2,2 114.68

43.46

0.72 0.47 0.26

2,2,3 2,2,4 2,2 101.95

36.45

0.62 0.36 0.13

2,2,3 2,3,3 2,2 99.77 35.80

0.69 0.42 0.15

2,2,3 2,2,2,2 2,2 90.07 29.63

0.48 0.19 0.04

Classical group screeningDistribution of S

Risk of exceeding a target

Choosing a strategy

• Is CGS a better option?

• IGS is more likely to exceed a target

Choosing a strategy – Simulation software

• Provides values of active and non-active effects sampled from specified distributions

• Forms the groups of factors at random

• For each selection of effect values, software simulates an experiment

• Calculates proportions of active individual main effects and interactions MISSED

• Process repeated 500 times

Assesses risk of missing active effects

Choosing a strategy - Simulation results

Strategy

CGS IGS

control main effects

6 – 17 1 – 11

control x control interactions

47 – 73 27 – 63

control x noise interactions

71 – 96 26 - 78

Percentage of active individual effects missed

Conclude: CGS misses more active individual effects than IGS

Software Results - CGSVery likely

control

Less likely

control

Noise

E(S)sd(S

)P(S>80)

P(S>110)

P(S>140)

Grouping

7 8 2,2 141.75

46.16

0.83 0.83 0.63

7 2,6 2,2 118.89

42.89

0.76 0.62 0.32

7 3,5 2,2 114.16

43.21

0.79 0.59 0.25

7 4,4 2,2 112.68

43.46

0.72 0.47 0.26

7 2,2,4 2,2 99.95 36.45

0.62 0.36 0.13

7 2,3,3 2,2 97.77 35.80

0.69 0.34 0.15

7 2,2,2,2 2,2 88.07 29.63

0.48 0.19 0.04

2,5 8 2,2 142.75

46.16

0.83 0.83 0.63

2,5 2,6 2,2 119.89

42.89

0.76 0.62 0.32

2,5 3,5 2,2 115.16

43.21

0.79 0.59 0.34

2,5 4,4 2,2 113.68

43.46

0.72 0.47 0.26

2,5 2,2,4 2,2 100.95

36.45

0.62 0.36 0.13

2,5 2,3,3 2,2 98.77 35.80

0.69 0.42 0.15

2,5 2,2,2,2 2,2 89.07 29.63

0.48 0.19 0.04

3,4 8 2,2 142.75

46.16

0.83 0.83 0.63

3,4 2,6 2,2 119.89

42.89

0.76 0.62 0.32

3,4 3,5 2,2 105.16

43.21

0.79 0.59 0.39

3,4 4,4 2,2 113.68

43.46

0.72 0.47 0.26

3,4 2,2,4 2,2 100.95

36.45

0.62 0.36 0.13

3,4 2,3,3 2,2 98.77 35.80

0.69 0.42 0.15

3,4 2,2,2,2 2,2 89.07 29.63

0.48 0.19 0.04

2,2,3 8 2,2 143.75

46.16

0.83 0383 0.63

2,2,3 2,6 2,2 120.89

42.89

0.76 0.62 0.32

2,2,3 3,5 2,2 116.16

43.21

0.79 0.63 0.34

2,2,3 4,4 2,2 114.68

43.46

0.72 0.47 0.26

2,2,3 2,2,4 2,2 101.95

36.45

0.62 0.36 0.13

2,2,3 2,3,3 2,2 99.77 35.80

0.69 0.42 0.15

2,2,3 2,2,2,2 2,2 90.07 29.63

0.48 0.19 0.04

Investigation of groupings

• 4 noise factors grouped in pairs

Very likely control

Less likely control E(S) sd(S) P(S>120) P(S>150) P(S>180)

Grouping

2,2,3

2,2,3

2,2,3

2,2,3

2,2,3

2,2,3

* 2,2,3

8

2,6

3,5

4,4

2,2,4

2,3,3

2,2,2,2

146.15

130.39

126.19

124.89

117.85

115.69

112.97

20.86

18.86

18.55

18.44

15.72

15.34

13.00

0.88

0.71

0.63

0.61

0.44

0.38

0.27

0.43

0.15

0.09

0.08

0.02

0.01

0.00

0.04

0.00

0.01

0.00

0.00

0.00

0.00

• Noise factors in two groups of size 2 was consistently more economical

Conclusions

• Good grouping strategies for minimising E(S):

– Use groups as small and as equal in size as possible

– Group together factors with higher probabilities of their main effects being active