DOE ApplicationsDOE Applications
Oct 16th 2002ASQ Section 702: San Gabriel Valley
byDr. Raj Palanna
Objective of Talk
Give a “flavor” for actual applications of DOE in industry
Provide an insight into how complex theoretical concepts are applied in real-world situations
Build on the ASQ 702 September presentation DOE - Basic Concepts by Dr. Kurt Palmer
Agenda
Deep Dive - 2 ExamplesQuick High Level Overview - Few
More ApplicationsQ&A
* These examples are taken from applications at different companies. Some of the data and facts have been “coded” to protect proprietary information.
1. ECS Pack Switch Failure InvestigationStarting February 2002 (until April
2002) a total of 21 switches have been rejected at XXX Aircraft Company’s aircraft test line, for failure to “open” at max. 200 deg. F.
Shipments of multi million dollar aircraft are at risk
Key Planning Aspects
Immediate containment actions takenTeams working at 3 tiers of supply chainEvaluate the “usual suspects”*
Testing setup and process Manufacturing process Application at system level, etc
Different schools of quality uses slightly different methods for this exercise
* This presentation will concentrate only on the DOE aspect of the problem
DOE Problem Statement
Parts that failed at customer passed at supplier test process. Test method is a suspect. A DOE was set up to understand the sensitivity of switches to the test parameters
6 Parts were chosen for DOE - 3 New, 3 Rejected
DOE Pre-planing
A complex DOE THOUGHT MAP was developed Factor Selection Response Selection Measurement Repeatability Studies Noise Impact Confounding Randomization Resource Logistics Management Approvals Observation/ Copious Notes etc
DOE Orthogonal Matrix Building Exercise
You Have Been Tasked to Optimize Gas Mileage on Your Car Team Needs to Test 4 Factors Has Resources for 8 Runs Res. IV Confounded Design in
Acceptable (Don’t worry if you don’t understand this bullet!!)
What is the DOE Matrix?
DOE Matrix
Factor A Factor B Factor C Factor D Response
Run # Test #
Depth of Switch (Plate) Ramp Rate
Oil Type/ Viscosity
(Flow)Mounting Technique
OutputSN 1
OutputSN 2
OutputSN 3
OutputSN 4
OutputSN 5
OutputSN 6
Average "Good"
Average "Rejected'
Average Total
1 1 Submerged 2DC 200 10CST Threaded 188.2 187.9 187.9 191.8 191.6 191.8 188.0 191.7 189.9
7 2Not
Submerged 2DC 200 10CST Set On 190.3 188.5 188.4 213.3 219.4 220.2 189.1 217.6 203.4
6 3 Submerged 10DC 200 10CST Set On 188.8 188.7 188.0 203.2 207.5 209.1 188.5 206.6 197.6
8 4Not
Submerged 10DC 200 10CST Threaded 188.7 188.0 187.3 195.3 200.6 200.5 188.0 198.8 193.4
3 5 Submerged 2DC 210 100CST Set On 192.8 190.6 190.8 215.6 222.0 222.0 191.4 219.9 205.6
5 6Not
Submerged 2DC 210 100CST Threaded 190.9 189.9 191.0 207.8 212.7 212.9 190.6 211.1 200.9
2 7 Submerged 10DC 210 100CST Threaded 189.0 187.9 189.3 204.4 213.7 213.8 188.7 210.6 199.7
4 8Not
Submerged 10DC 210 100CST Set On 199.6 191.6 195.4 253.1 246.4 250.8 195.5 250.1 222.8
24-1ResIV
0 10 20
AC
AB
B
AD
A
C
D
Pareto Chart of the Effects(response is REJECTED, Alpha = .30)
A: PlateB: Ramp RatC: Oil TypeD: Mounting
-20 -10 0 10 20
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
Effect
Nor
mal
Sco
re
C
D
Normal Probability Plot of the Effects(response is REJECTED, Alpha = .30)
A: PlateB: Ramp RatC: Oil TypeD: Mounting
DOE Analysis:“Rejected” Switches Average Trigger Temperature
Plate Ramp Rate Oil Type Mounting
Not Submerged
Submerged
2 10 DC 200 10CST
DC 210 100CST
Set On
Threaded
204
209
214
219
224
RE
JEC
TED
Main Effects Plot - Boeing Rejected Switches (F)
Type of TESTING OIL (C) influences test results and Type of MOUNTING (D) influences test results
2 10 DC 200 10C
DC 210 100
Set OnThreaded
190
200
210
190
200
210
190
200
210Plate
Ramp Rate
Oil Type
Mounting
Not Submer
Submerged
2
10
DC 200 10C
DC 210 100
Interaction Plots - Boeing Rejected Switches (F)
DOE Analysis:“New” Switches Average Trigger Temperature
0 1 2 3
B
AD
AC
AB
A
D
C
Pareto Chart of the Effects(response is GOOD, Alpha = .30)
A: PlateB: Ramp RatC: Oil TypeD: Mounting
-3 -2 -1 0 1 2 3
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
Effect
Nor
mal
Sco
re
C
Normal Probability Plot of the Effects(response is GOOD, Alpha = .30)
A: PlateB: Ramp RatC: Oil TypeD: Mounting
2 10 DC 200 10C
DC 210 100
Set OnThreaded
188.0
190.5
193.0
188.0
190.5
193.0
188.0
190.5
193.0Plate
Ramp Rate
Oil Type
Mounting
Not Submer
Submerged
2
10
DC 200 10C
DC 210 100
Interaction Plots - 'Good' Switches (F)
Plate Ramp Rate Oil Type Mounting
Not Submerged
Submerged
2 10 DC 200 10CST
DC 210 100CST
Set On
Threaded
188.5
189.3
190.1
190.9
191.7
GO
OD
Main Effects Plot - 'Good' Switches (F)
No Impact of Factors A, B, C and D on “New” Switches
180 185 190 195 200 205
LSL USL
Test Parameters Outside ATP - Trigger Temp Spread 14353
USL
Target
LSL
Mean
Sample N
StDev (ST)
StDev (LT)
Cp
CPU
CPL
Cpk
Cpm
Pp
PPU
PPL
Ppk
PPM < LSL
PPM > USL
PPM Total
PPM < LSL
PPM > USL
PPM Total
PPM < LSL
PPM > USL
PPM Total
200.000
*
180.000
191.037
8
2.81155
3.90992
1.19
1.06
1.31
1.06
*
0.85
0.76
0.94
0.76
0.00
0.00
0.00
43.23
716.94
760.17
2379.21
10945.68
13324.89
Process Data
Potential (ST) Capability
Overall (LT) Capability Observed Performance Expected ST Performance Expected LT Performance
STLT
185 187 189 191 193
LSL USL
Test Parameters Outside ATP - Trigger Temp Spread 14412
USL
Target
LSL
Mean
Sample N
StDev (ST)
StDev (LT)
Cp
CPU
CPL
Cpk
Cpm
Pp
PPU
PPL
Ppk
PPM < LSL
PPM > USL
PPM Total
PPM < LSL
PPM > USL
PPM Total
PPM < LSL
PPM > USL
PPM Total
200.000
*
180.000
189.137
8
1.32979
1.45172
2.51
2.72
2.29
2.29
*
2.30
2.49
2.10
2.10
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
Process Data
Potential (ST) Capability
Overall (LT) Capability Observed Performance Expected ST Performance Expected LT Performance
STLT
180 185 190 195 200
LSL USL
Test Parameters Outside ATP - Trigger Temp Spread 15235
USL
Target
LSL
Mean
Sample N
StDev (ST)
StDev (LT)
Cp
CPU
CPL
Cpk
Cpm
Pp
PPU
PPL
Ppk
PPM < LSL
PPM > USL
PPM Total
PPM < LSL
PPM > USL
PPM Total
PPM < LSL
PPM > USL
PPM Total
200.000
*
180.000
189.762
8
1.65907
2.74439
2.01
2.06
1.96
1.96
*
1.21
1.24
1.19
1.19
0.00
0.00
0.00
0.00
0.00
0.00
187.37
95.61
282.98
Process Data
Potential (ST) Capability
Overall (LT) Capability Observed Performance Expected ST Performance Expected LT Performance
STLT
150 170 190 210 230 250 270
LSL USL
Test Parameters Outside ATP - Trigger Temp Spread 14954
USL
Target
LSL
Mean
Sample N
StDev (ST)
StDev (LT)
Cp
CPU
CPL
Cpk
Cpm
Pp
PPU
PPL
Ppk
PPM < LSL
PPM > USL
PPM Total
PPM < LSL
PPM > USL
PPM Total
PPM < LSL
PPM > USL
PPM Total
200.000
*
180.000
210.562
8
15.1596
19.6889
0.22
-0.23
0.67
-0.23
*
0.17
-0.18
0.52
-0.18
0.00
750000.00
750000.00
21897.23
757021.74
778918.98
60298.58
704183.21
764481.79
Process Data
Potential (ST) Capability
Overall (LT) Capability Observed Performance Expected ST Performance Expected LT Performance
STLT
150 170 190 210 230 250 270
LSL USL
Test Parameters Outside ATP - Trigger Temp Spread 15007
USL
Target
LSL
Mean
Sample N
StDev (ST)
StDev (LT)
Cp
CPU
CPL
Cpk
Cpm
Pp
PPU
PPL
Ppk
PPM < LSL
PPM > USL
PPM Total
PPM < LSL
PPM > USL
PPM Total
PPM < LSL
PPM > USL
PPM Total
200.000
*
180.000
214.238
8
14.0578
16.9269
0.24
-0.34
0.81
-0.34
*
0.20
-0.28
0.67
-0.28
0.00
875000.00
875000.00
7435.83
844418.92
851854.75
21553.66
799858.81
821412.47
Process Data
Potential (ST) Capability
Overall (LT) Capability Observed Performance Expected ST Performance Expected LT Performance
STLT
150 170 190 210 230 250 270
LSL USL
Test Parameters Outside ATP - Trigger Temp Spread 14935
USL
Target
LSL
Mean
Sample N
StDev (ST)
StDev (LT)
Cp
CPU
CPL
Cpk
Cpm
Pp
PPU
PPL
Ppk
PPM < LSL
PPM > USL
PPM Total
PPM < LSL
PPM > USL
PPM Total
PPM < LSL
PPM > USL
PPM Total
200.000
*
180.000
215.137
8
14.7670
18.1465
0.23
-0.34
0.79
-0.34
*
0.18
-0.28
0.65
-0.28
0.00
875000.00
875000.00
8668.88
847340.05
856008.93
26414.19
797910.71
824324.90
Process Data
Potential (ST) Capability
Overall (LT) Capability Observed Performance Expected ST Performance Expected LT Performance
STLT
Switches from New Batch
Switches from Customer ‘Rejected” Batch
Spread of Results Under Different Testing Conditions
Higher Spread Has been Observed in “Rejected Switches”
ITT ATP Data 641121, Jan 1999 - Mar 2002 (Actuation)
170
175
180
185
190
195
200
205
210
1
134
267
400
533
666
799
932
1065
1198
1331
1464
1597
1730
1863
1996
2129
2262
2395
2528
2661
2794
2927
3060
3193
3326
3459
3592
3725
3858
3991
4124
4257
4390
4523
4656
4789
Test #
Deg
ree
F
High Variation?
Historical Review of Test Data to Evaluate Process
Conclusions
A particular batch of switches were found to be in under-fill condition and switches became “sensitive” to testing parameters/ operating conditions and lost its robustness to environment
Manufacturing process was fixedTest was augmented to prevent this
situation from happening again
2. Aircraft Motor Performance OptimizationProblem statement: shop failures at
test. Brake was releasing under the “brake release voltage specification”
Evaluate the “usual suspects” Testing setup and process Manufacturing process Application at system level, etc
Developed a DOE thought map
DOE Matrix .. MotorInput Factors Output
Average
Run # Kit #Static Brake
CW
Static Brake CCW
Static Brake(in-oz)
6 1 -1 0.00019 -1 0.0008 -1 0.003in -1 No -1 0.003in 18.23 15.7 16.965
4 2 1 0.00043 -1 0.0008 -1 0.003in -1 No 1 0.005in 9.12 9.26 9.19
1 3 -1 0.00014 1 0.0003 -1 0.003in -1 No 1 0.005in 8.35 9.26 8.805
7 4 1 0.00051 1 0.0003 -1 0.003in -1 No -1 0.003in 8.42 8.34 8.38
5 5 -1 0.0002 -1 0.0006 1 0.005in -1 No 1 0.005in 9.06 9.42 9.24
2 6 1 0.00037 -1 0.0006 1 0.005in -1 No -1 0.003in 9.09 8.6 8.845
8 7 -1 0.00016 1 0.0003 1 0.005in -1 No -1 0.003in 7.33 7.32 7.325
3 8 1 0.0004 1 0.0005 1 0.005in -1 No 1 0.005in 9.3 9.18 9.24
14 1-2 -1 0.00019 -1 0.0008 -1 0.003in 1 Yes -1 0.003in 22.03 19.38 20.705
12 2-2 1 0.00043 -1 0.0008 -1 0.003in 1 Yes 1 0.005in 18.65 18.13 18.39
9 3-2 -1 0.00014 1 0.0003 -1 0.003in 1 Yes 1 0.005in 14.17 13.3 13.735
15 4-2 1 0.00051 1 0.0003 -1 0.003in 1 Yes -1 0.003in 9.87 9.72 9.795
13 5-2 -1 0.0002 -1 0.0006 1 0.005in 1 Yes 1 0.005in 17.12 17.21 17.165
10 6-2 1 0.00037 -1 0.0006 1 0.005in 1 Yes -1 0.003in 15.38 14.85 15.115
16 7-2 -1 0.00016 1 0.0003 1 0.005in 1 Yes -1 0.003in 15.54 14.76 15.15
11 8-2 1 0.0004 1 0.0005 1 0.005in 1 Yes 1 0.005in 14.36 13.45 13.905
D=ABCEnd Play
ADisk Flatness
BArmature TIR
CBrake Gap
ERun In
25-1ResIV
DiskFlat ArmTIR Brake Gap Run-In End Play
-1 1 -1 1 -1 1 -1 1 -1 1
10.0
11.2
12.4
13.6
14.8
Sta
tic B
rake
Main Effects for DOE Part 2
Use : • Run-In• High Armature TIR• Flat Disk
-1 1 -1 1 -1 1 -1 1
8
13
18 8
13
18 8
13
18 8
13
18DiskFlat
ArmTIR
Brake Gap
Run-In
End Play
-1
1
-1
1
-1
1
-1
1
Interaction Plot for DOE 2
With Low Disk Flatness Use 3 mm (Low) End-Play.
0 1 2 3 4 5
E
AD
ADE
ACD
CD
DE
BD
AB
C
ABD
AC
A
AE
B
D
Pareto Chart of the Effects(response is Static B, Alpha = .30)
A: DiskFlatB: ArmTIRC: Brake GaD: Run-InE: End Play
-4 -3 -2 -1 0 1 2 3 4 5 6
-1
0
1
Effect
Nor
mal
Sco
re
D
AE
A
B
Normal Probability Plot of the Effects(response is Static B, Alpha = .30)
A: DiskFlatB: ArmTIRC: Brake GaD: Run-InE: End Play
Test for Statistical Significance
Results Are Statistically Significant
2 4 6 8 10 12 14 16
-1
0
1
Observation Order
Res
idua
l
Residuals Versus the Order of the Data(response is Static B)
10 15 20
-1
0
1
Fitted Value
Res
idua
l
Residuals Versus the Fitted Values(response is Static B)
-1 0 1
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
Nor
mal
Sco
re
Residual
Normal Probability Plot of the Residuals(response is Static B)
-1.5 -1.0 -0.5 -0.0 0.5 1.0 1.5
0
1
2
3
4
Residual
Fre
que
ncy
Histogram of the Residuals(response is Static B)
Randomness Test
There is No Unusual Observations
Conclusions - Motor DOE
Experiment showed statistical significance.
Settings for Static BrakeBrake Cycling….Introduce Brake Cycling in MOT-
ChuckHigh Armature TIR.???? Do we ask for “bad parts”?Flat Disk….Within 1/10th..Need Process ControlWith Low Disk Flatness Use 3 mm(Low) EndPlay.
(With Flat Disk Set Low EndPlay). Change MOT - Chuck.
This will improve Static Brake Problems.
3. Nozzle Air Flow Optimization DOE
Problem statement: air cycle machine nozzle hole-to-hole variation need to be minimized to improve system performance
Process: multi spindle drillingDOE response: diameter sigma between
different holes in a single nozzleFactors: machining factors
Reamer speed, Reamer feed, Reamer diameter, Stock (after drilling)
4. Vapor Cycle Pack Bearing Failure Investigation
Problem statement: bearing failures of VCS pack on aircraft
Type of DOE: design robustness evaluation
Response: measure of fatigue on the bearing after specified run time
DOE factors: product design variables
5. DOE for Car Door Panel Delimitation Problem
Problem statement: delaminating of vinyl from the foam substrate of an car door interior
Process: door panel molding processResponse: lamination evaluation on a likert
scale + measure of area of delaminatingDOE factors: molding machine factor
Overall temp, Heat upper, Heat lower, Oven temp, Backing type, Density, Vacuum pressure, Vacuum delay
6. Actuator Pin Alignment DOE
Problem statement: aircraft door actuator connector pin alignment out of specification
Type of DOE: combination assembly process evaluation plus design features evaluation
DOE response: distance from nominal in circular domain
DOE factors: assembly technique variables plus part features Twist, Length, Sleeve
Practice Idea:Gas Mileage Optimization DOE Problem statement: pick a response variable
to optimize on the car (gas mileage…) Choose factors that might influence this output
response variable Setup the DOE, run the DOE, analyze DOE Other ideas:
Optimize volume of popcorn produced Etc……. Any suggestions from audience?
DOE Is Not the Easiest Application Tool Around Town….But If You Master DOE, You Will Be a Champion QE!!
Practice, Practice, Practice
Appendix
Reference Books Douglas C. Montgomery: Design and Analysis of
Experiments Ronald D. Moen Et Al.: Improving Quality Through Planned
Experimentation Madhav S. Phadke: Quality Engineering Using Robust
Design
Software Minitab JMP Etc