Doe As Process Control Introduction

Using Designed Experimentation as a practical introduction to Process

Control in a Thermal Spray Shop

Kelly D. Brown, Ph.D. MBBAvalanche Process Improvement, LLC

John Sauer, PESauer Engineering

September 10, 2010

CONSTANTS:- Powder Feed Rate- Turntable RPM- Gun Type

NOISE FACTORS:- Powder Particle Size- Booth Air Flow- Spray Angle

CONTROL FACTORS:- Amperage- Gas Flow- Air/Fuel Ratio- others…

RESPONSES:- Film Thickness - hardness- % oxides- others…

Deposit Metal Film

Change Control

Monitor

Measure

Model TS Shop Process

SelectRecipe

Spray Part

ShopMetrics

ShopMeasures

Process Control in 2 Parts

Part 1 – Process Control Concepts and example of Thermal Spray troubleshooting with Root Cause Analysis

Part 2 – Advanced Process Control using Designed Experimentation (DOE) in a Thermal Spray operation

Model TS Shop Process

SelectRecipe

Spray Part

ShopMetrics

ShopMeasures

CONSTANTS:- Powder Feed Rate- Turntable RPM- Gun Type

NOISE FACTORS:- Powder Particle Size- Booth Air Flow- Spray Angle

CONTROL FACTORS:- Amperage- Gas Flow- Air/Fuel Ratio- others…

RESPONSES:- Film Thickness - hardness- % oxides- others…

Deposit Metal Film

Change Control

Monitor

Measure

Process Control Concepts for a Thermal Spray Shop

Metrics are a product of the process design, measures and inputs and outputs used to spray parts. – Measures – Part quantity, specifications, part prep, and more…– Metrics – Production output, cycle time, quality, and more…– Process Inputs – Gas flows, current, voltage, …– Process Outputs – part temp, film thickness, microstructure, …– Process Design – arrangement of the process steps

Model TS Shop Process

SelectRecipe

Spray Part

ShopMetrics

ShopMeasures

Thermal Spray Shop Controls

Process Control Facts - 1

– Problems can result from errors in Measures, Inputs, Outputs, or Metrics. – Problems can be caused, hidden, repaired by Process Design

– Outputs become inputs for downstream processes– Problems with inputs expand, or can “go underground”

– Process design usually determines the speed and throughput of the shop

– Process inputs and outputs usually determine quality of part– Poor quality “spills over” as a speed and throughput problem

Model TS Shop Process

SelectRecipe

Spray Part

ShopMetrics

ShopMeasures

Thermal Spray Shop Controls

Process Control Facts - 2Processes operate with inputs and outputsProblems are generally reported as metrics

Root cause of process problems not resolved with metrics

Must locate problem in the process to formulate best solution

Model TS Shop Process

SelectRecipe

Spray Part

ShopMetrics

ShopMeasures

Root Cause Analysis aka, RCA or 5-Why

analysis.

Why?

You keep asking “Why?” until you are

exhausted!!!

An experienced facilitator can help your organization learn to do

“5-why” right

First Step – Root Cause Analysis

RCA IS A SIMPLE CONCEPT:1. Define the Problem2. Define the Process3. Start asking why until one or more actionable causes are

determined4. Generate permanent corrective actions5. Document and track results

Why?

Why?

Why?

Why?

Why?

Why?

PROBLEM:50% failure rate

from inconsistent large un-melts found

in T800 coatings

Using Root Cause Analysis to Identify Best Fit Solutions & DOE’s

Root Cause Analysis (5-Why) as tool to focus potential DOE improvements

• Not all problems need DOE to resolve• Orient DOE testing in area likely be responsible for failures• Manage resource to resolve most important challenges first

Powder ?Plasma?

Spec’s

?

RCA: 5-Why Analysis of T800 failure with Actionable Solutions

Model TS Shop Process

SelectRecipe

Spray Part

ShopMetrics

ShopMeasures

High percentage of 006G & 006S microstructure

unmelts

Why? Unmelts and Oxides

Why?

Large Particles visible in

Micrographs

Large Particles in Powders

Specifications not available on

floor

1. Specifications not acknowledged

by vendors

Why?

3. Powder Surging during Plasma Spray

Why?

Engineering validation OK’s

particle size variance

Why?

Good materials are not available

Vendors not asked to fix problems

Need production right away

Why?Insufficient heat

to melt large particles

Why?

Why?

Why?

2. Incorrect Spray Parameters?

Problems Need The Right TeamWhat You Can Do Depends on What Team You Are On

Design (Owned by Management / Engineering)Equipment Selection Process Steps / Sequence

Shop Measures (Owned by customer)Specifications Quantity / type of parts

Process Inputs (Owned by Management / Engineering)Gas flows Power settingsGas pressures Part setup

Process Outputs (Controlled by design, measures and process inputs)Velocity Temperature Metallography Tensile

Metrics ( Owned by Customers / Management)Throughput Quality / SpeedCost

RCA Results – Multiple Problems

Insufficient heat to melt large

particles1. Powder Size Problem:

NOT DOE – Process Design Problem, possible problem with specificationsChange process design & resolve issues with vendors / specifications

2. Spray Parameter Problem:DOE – Process Input Problem,Unmelts can result from low particle temperature and velocity, driven by spray parameter inputs

3. Powder Surging Problem:Possible DOE – May be related to maintenance or to powder delivery and gas flow settings

1. Specifications requirement not

delivered by buyers

2. Incorrect Spray Parameters?

3. Powder Surging during Plasma

Spray

Problem Location determines solution type

Thermal Spray Inputs & OutputsQUIZ:

– How many of these parameters to you monitor in your shop?

– What are the best setting for T-800?– Will more than one set of parameters work well for T-

800?– What is the optimal settings for T-800, or any other

coating?– How would you go about finding the best settings?Sec. 1: Part Identify and Technical Reference Information: Sec. 3: Gun Setup Parameters:

Part Powder Spec. Gun Type # of Air JetsRobot Recipe Powder Lot No Last Gun Rebuild Location of Air JetsGun Recipe Powder manufacturer Port Diam. (typ. 1.8mm) Cross distancePowder Type Part Surface ID Port Dist. (typ. 6mm) Inj Loc. (down = 12:00)Powder Recipe Inj Port Angle (90° or 105°)

Sec. 4: Part & Test Coupon Setup ProgramSpray Distance (in) Booth Air Flow (LFM)

Sec. 2: Spray Parameters (Plume critical inputs): Spray Diameter (in) Cooling air Location(s)Target Actual Target Actual Turntable RPM

Ar SCFH Feed Rate Gun Travers Rate Coupon OrientationH2 SCHF RPM Spray Angle (verticle)Ctrl Type CG SCFH Spray Angle (horizontal)Current Block Type Sec. 5: Available Measurements and Tests for Booth equipment and POSPower Last Hopper Rebuild Intensity # Traverses & PassesP.S. Volts Velocity Dep. ThickGun Volts Temp. Dep Rate

Plume Height Lab Qual IDPlume Width Lab Status

DOE is the ideal tool for answering Thermal

Spray questions like these with known

certainty

?

More Process Control facts for Thermal Spray Shops

Broken equipment produces broken results Process design inputs must be held constant if

you want a consistent outcome

You can “see” a lot more than you can measure You already know a lot more than you use about your process

Not everything that you can measure is important Many important things are difficult to measure Good measurements require skill, repeatable measurements discipline

Environmental noise in your shop every day. It is not a LAB! There are many possible parameters but a few are stronger than the rest If strong parameters are set correctly, minor parameters are easier (sweet

spot)

Parts that closely related can be processed with similar or the same process parameters

Some technologies are easier than others. Sometimes you get lucky, but your customers are not big fans of luck.

The Answer

A Practical Design of Experiment (DOE) Method

CONSTANTS: - Powder Feed Rate - Turntable RPM - Gun Type

NOISE FACTORS: - Powder Particle Size - Booth Air Flow - Spray Angle

CONTROL FACTORS: - Amperage - Gas Flow - Air/Fuel Ratio - others…

RESPONSES: - Film Thickness - hardness - % oxides - others…

Deposit Metal Film

Change Control

Monitor

Measure

Why DOE testing for Plasma?

DOE is the recognized “World Class” approach for quickly solving difficult technical problems that have multiple controlling factors.

Many “difficult” problems can be solved quicker with simpler quality tools, however the DOE planning process also uncovers and solves many of the same problems.

DOE offers a “fine-tuning” approach for determination of the optimal combinations of thermal spray parameters

DOE is empirical, well suited to work in production environments and providing answers useful in realistic process environments

Progression of Engineering Tests

Cargo Cult - A persistent belief that wealth and beneficialresults come from spiritual means, deities, and ritualbehavior, in spite of evidence to the contrary.

A common human behavior when confronted by advanced technology (see Cargo Cult Science, Pseudoscience).

Trial and Error - an attempt to achieve a good process output based on intuition, guesses, and repeated testing. Often inspired but seldom repeatable.

OFAT - One Factor At a Time - an attempt to define process behavior “one factor at a time”. Marginally more effective than trial and error, but conceptually blind to effect of one factor other active factors.

DOE - Design of Experiments - A predetermined series of runs in which multiple factors are evaluated simultaneously to determine their individual and combined influence on the response.

Will DOE solve all my Thermal Spray Problems?

One method does not work everywhere on every issue

Sometimes problems are easier than “DOE” solutions

Computers are great but we don’t need a computer when the only step is adding 2 numbers together…..

Problems are usually complex and the RCA/DOE approach can identify what needs DOE help and what can be solved with other methods

DOE Benefits

DOE provides confident solutions for processes with multiple controlling factors, something other experimental approaches do not deliver.

DOE planning and execution is a great learning tool - teaching fundamental principles of measurement and process control to a process area

Identification of “easy” solutions often become obvious during the planning phase

DOE execution improves your ability to measure and confidence in your process results

DOE solutions can be expressed as simple test results, graphical analysis or empirical models with known confidences depending on organizational need.

The Language of DOE

Control factor - Numerical or categorical inputs used to control process response.

Response factor - A numerical process output.Constants - A process input held constant during the DOE test.Noise factor - Nuisance factor, typically too expensive, difficult or dangerous

to controlP-Map - A conceptual map showing the relationship of process factors to

responsesA P-Map used as a shorthand description for a Plasma Spray DOE

CONSTANTS: - Powder Feed Rate - Turntable RPM - Gun Type

NOISE FACTORS: - Powder Particle Size - Booth Air Flow - Spray Angle

CONTROL FACTORS: - Amperage - Gas Flow - Air/Fuel Ratio - others…

RESPONSES: - Film Thickness - hardness - % oxides - others…

Deposit Metal Film

Change Control

Monitor

Measure

Two Different DOE ApproachesProject Model:

• DOE is ran as a large “stand-alone” project with well defined goals, resources, roles, responsibilities and a DOE consultant to direct the planning and testing. • Typically large, expensive tests and validation runs. • A good way to introduce “DOE” to a large organization.• Consultant responsible for DOE selection, calculations, interpretations.

COACH DESIGNS DOE AND PUSHES IT THROUGH ORGINIZATION

Two Different DOE Approaches

Operational (training) Model:• The DOE planning sequence applied to a generic technical problem with the goal of improving the overall Engineering response to technical problems.

• Many small problems are solved in the DOE planning stage, and DOE tests are a series of small tests that progressively build knowledge and confidence of a company’s technical resource base.

• DOE coach operates as EngineeringInstructor helping internal resources plan and run tests.

Is One Better Than the Other??

We need a PRACTICAL approach to DOE!

A Practical DOE approach

DOE planning sequence:

1. State the objective2. Define the response3. Define control and noise factor4. Select appropriate DOE matrix5. Consider experimental error6. Evaluate the measurement system7. Run the DOE8. Analyze the results9. Predict preferred response10. Confirmation run11. Implementation

DOACT

(1-5)

(11)

(10)

PLAN

CHECK

(6-9)

Planning and preparation is critical to DOE success.

Without planning, the probability of running the right DOE is small and the probability of excessive experimental error is large.

DOE’s require Measurement TestsA simple measurement test on coating thickness (15 minutes to

complete)

Test Coupon Thickness Test

#1

#2

#3

#4 #5

repeat 1 repeat 2746 745748 743730 729743 747744 745

Test Coupon Thickness Test

#6

#7

#8

#9 10

repeat 1 repeat 2717 726710 708707 705706 702702 703

Standard error is analogous to Standard Deviation. A single micrometer reading will be within 5.5 mils of the true value 95% of the time.

Simple Measurement EvaluationSx repeat 1 repeat 2 Average err1 err21 746 745 745.5 0.5 -0.52 748 743 745.5 2.5 -2.53 730 729 729.5 0.5 -0.54 743 747 745 -2 25 744 745 744.5 -0.5 0.56 717 726 721.5 -4.5 4.57 710 708 709 1 -18 707 705 706 1 -19 706 702 704 2 -2

10 702 703 702.5 -0.5 0.5

sumsqerr 75 =SUMSQ(err1 + err2 + ...)

dof 10 = 20 data points - 10 averages

Standard Error 2.74 =SQRT(Sumsqerr/dof)

95% CI 5.5 =+/- 2*Standard Err

Change these parameters: To control these response:

DOE test to quantify temperature control forprimary, secondary gasand power settings

DOE test result for Temperature

Factor Table

Level PRI FLOW SEC FLOW POWER

Low 60 22 37Mid 70 26 39High 80 30 41

Accura Spray Results

INTENSITY VELOCITY TEMP HEIGHT WIDTH

FLAME TEMP

2600

2650

2700

2750

2800

60 70 80 22 26 30 37 39 41

PRI FLOW SEC FLOW POWER

More DOE responses

Test Matrix and Run ResultsTest Plan Test Results - Accuraspray Test Results - Deposition

RunPRI FLOW SEC FLOW Power AMPS INTENSITY VELOCITY TEMP Height Width Thickness

Surface Roughness

1 70 26 39 650 113 113 2672 6.27 11.7 0.01146 358.72 60 22 37 650 111 101 2578 5.74 13.1 0.0108 398.73 80 22 37 650 74 133 2589 4.43 10.7 0.0108 434.34 60 30 37 650 143 98 2723 7.17 13.2 0.0112 379.85 80 30 37 650 80 131 2639 4.96 10.5 0.01212 390.36 60 22 41 650 126 106 2710 6.34 13 0.0067 408.27 80 22 41 650 86 140 2672 4.95 11 0.01032 417.08 60 30 41 650 173 104 2900 8.45 14.1 0.01058 396.79 80 30 41 650 91 137 2804 4.89 12.1 0.01048 432.0

10 70 26 39 650 102 119 2699 4.74 15 0.01042 356.3Average 109.9 118.2 2698.6 5.8 12.4 0.0105 397.2Est Err 11 6 27 1.53 3.3

Empirical Models

2600

2700

2800

6070

2800

2900

28.5

27.080

30.0

28.5

31.5

30.0

TEMP

Power (kw)

Pri Flow

Surface Plot of TEMP vs Power (kw), Pri Flow

TEMP = 2696 -25.9*PRI_FLOW + 64.6*SEC_FLOW + 69.6*POWER

2600

2700

2800

212124

27

2800

2900

28.5

27.030

30.0

28.5

31.5

30.0

TEMP

Power (kw)

Sec Flow

Surface Plot of TEMP vs Power (kw), Sec Flow

Test matrix contains 70 independent process results, providing average responses and 63 opportunities to gain knowledge about the process.

Increasing the electrical power to the gun strongly increases flame temperature. The temperature is moderately sensitive to secondary flow and weakly sensitive to primary flow.

Quantitative equation providing quantitative predictions for range of control settings.

Graphical Summary of Accuraspray Data

VELOCITY

80

90

100

110

120

130

140

60 70 80 22 26 30 37 39 41

PRI FLOW SEC FLOW POWER

INTENSITY

60

80

100

120

140

160

60 70 80 22 26 30 37 39 41

PRI FLOW SEC FLOW POWER

TEMP

2550

2600

2650

2700

2750

2800

60 70 80 22 26 30 37 39 41

PRI FLOW SEC_FLOW POWERThickness

0.008

0.009

0.010

0.011

0.012

60 70 80 22 26 30 37 39 41

PRI FLOW SEC FLOW POWER

Roughness

320

340

360

380

400

420

440

60 70 80 22 26 30 37 39 41

PRI FLOW SEC FLOW POWER

A graphical guide for adjusting Accuraspray outputs, film thickness

and roughness.

Forced Ranking vs Accuraspray

Test Plan Test Results - Accuraspray

RunPRI

FLOWSEC

FLOWPower AMPS

INTENSITY

VELOCITY TEMP Height Width

A&B 1 70 26 39 650 113 113 2672 6.27 11.7C&D 2 60 22 37 650 111 101 2578 5.74 13.1E&F 3 80 22 37 650 74 133 2589 4.43 10.7G&H 4 60 30 37 650 143 98 2723 7.17 13.2I&J 5 80 30 37 650 80 131 2639 4.96 10.5K&L 6 60 22 41 650 126 106 2710 6.34 13M&N 7 80 22 41 650 86 140 2672 4.95 11O&P 8 60 30 41 650 173 104 2900 8.45 14.1Q&R 9 80 30 41 650 91 137 2804 4.89 12.1S&T 10 70 26 39 650 102 119 2699 4.74 15

Average 109.9 118.2 2698.6 5.8 12.4Est Err 11 6 27 1.53 3.3

Test Plan Test Results - Metallurgical Analysis

RunPRI

FLOWSEC

FLOWPower AMPS Rank 90 Rank 45

Oxide Level

% Unmelts % porosity Thickness

A&B 1 70 26 39 650 4 6 12 6% 3% 11C&D 2 60 22 37 650 9 4 11 4% 2% 11E&F 3 80 22 37 650 5 3 11 4% 2% 11G&H 4 60 30 37 650 6 10 13 4% 3% 10I&J 5 80 30 37 650 3 7 17.5 6% 2% 11K&L 6 60 22 41 650 8 5 12 3% 2% 10M&N 7 80 22 41 650 2 1 19 7% 2% 11O&P 8 60 30 41 650 10 8 12 4% 2% 10.5Q&R 9 80 30 41 650 1 2 14.5 5% 2% 10S&T 10 70 26 39 650 7 9 11.5 3% 2% 10

Velocity NOT Temperature found to be most control

factor

“Best” micrographs were qualified by forced ranking by two independent judges.

Test results used to identify “optimal” intensity, velocity and temperature settings.

Best results found to be the highest velocity settings.

Good micrographs at both 90 and 45 degree spray angles

Cold

Hot

FAST

FAST

DOE Planning Process – Adaptation for Powder Flow Optimization

Set Goal - Reduce variation in powder flow across range of delivery ratesDefine output measure - Powder delivery rate (gm/min)• Develop a measurement with known error. This calls for improvement in traditional “grab and bag”

powder samples.

Identify potential control parameters• Potential Control Parameters - Carrier Gas Flow, Injector Location, Injector Distance,

Injector Blocks, • Potential Signal Factors - Powder Feed Rate (demand) • Potential Noise factors - Powder Lot (particle size, carrier gas, short/long

powder tubing)

Select DOE matrix • Number of factors• Available test resource• Ease / difficulty for maintaining control

Determine minimum detection limit needed for successful testsRun measurement test and calculate minimum detectable difference• Improve measurement system as required or use replicate tests to reduce measurement

uncertainty

Trial run with “best case / worse case to validate measurement and response signals - Complete DOE if test response is sufficiently large.

Calculate DOE response, prepare graphs, empirical equations, validate with existing knowledge

Calculate “optimal setting”Complete validation with new “optimal setting”, determine experimental

error and likelihood of additional unknown factorsImplement new optimal parameter settings

Example of a DOE

Execution Plan for reducing Powder

Surges in Plasma Spray

Process

Powder flow under controlx x

x

x x x x

x xx

Inconsistent and

unpredictable powder flow

x

x x x x

xx

xxx xxx

RCA: Inconsistent coating quality due to

powder “spurts” during coating

Conclusions

•Process Control is a broader subject than DOE

•Process Control precedes DOE•DOE provides quantitative parameter optimization

•Running a DOE program is a practical way to:–Learn about process control–Test your measurement systems–Understand impact of one change on another–Lessen impact of environmental noise on shop quality– Improve your “Engineering testing” for problem solving–Put into practice shop floor knowledge not currently being

used