Robinson Presentation Slides 2.3.08

A Level 4 Evaluation of a Structured On-the-Job Training

Intervention at a North American Paper Mill

Pamela L. Robinson, Ph.D.

Background

• Quality Issues– Increase in customer returns/complaints

• Training Issues– No training center– No training coordinator– No up-to-date operator manuals

– No structured program

Needs Assessment

• Obtained support of senior management• Assessment developed

– Other training coordinators input– Management input

• Assessment completed– Production workers– Mill management

• Assessment Reviewed– Management meeting

Items Identified

• Quality concerns – Increased number of customer

returns/complaints

• Lack of effective operator training

• Lack of operator manuals

• Lack of training center and training materials

Program Developed

• Driven by identified objectives

• Developed by training team– Division training coordinator

– Mill training coordinators (SME’s)– Mill superintendents– Quality engineer

– Organizational Development

• Utilized input from operators

Evaluation Conducted

• Level 1– Survey of Participants

• Level 2– Course Tests Results– Checklist Completion

• Level 3– Observations and Sign-Offs by Supervisors

• Level 4– Analysis of Operator-Controllable Error Metrics– Survey of Training Team to Identify Success Factors

Questions

• Did training have desired effect on specified quality metrics?

• What factors contributed to success or failure of training?

Isolating the Effects of Training

• Metrics– Operator controllable errors

• Operator positions– Back or “dry” end

• Time periods – Assignment changes

• Data– Total– Machine– Team

The dry end operator-controllable defects

• Bad slit/not slit/poor slit quality• Improper/poor splicing• Incorrect quantity• Incorrect slit width• Incorrect stencil/label/packaging• Rolls not separated • Soft rolls/loose winds/telescoped• Wrong roll diameter

Excluded Factors

• Quality of raw materials

• Maintenance

• Front or “wet” end operator positions

Excluded Defects/Complaints

• Slime spots on paper

• Damage in warehouse

• Damage in shipment• Late shipment

The Results

What the Metrics Revealed

Training

0

10

20

30

40

50

60

70

80

90

Period 1

Period 6

Period 1 59.3 64.4

Period 6 76.7 85.5

% Certified % Certified/Some Training

OCE Metrics

0

20000

40000

60000

80000

100000

120000

Pre Training

Post Training

Pre Training 115390 25086.58

Post Training 63606 16633.24

OCE Lbs OCE $

The Rest of the Story

Looking Beyond the Obvious

Training

0

10

20

30

40

50

60

70

80

90

Period 1

Period 6

Period 1 59.3 64.4

Period 6 76.7 85.5

% Certified % Certified/Some Training

0

20

40

60

80

100

120

% Certified

% w/Training

% Certified 59.3 70.6 81.6 73.2 83.7 76.7

% w/Training 64.4 82.4 95.9 85.7 93.9 85.5

1 2 3 4 5 6

Quality Metrics

0

5000

10000

15000

20000

25000

30000

OCE Lbs

OCE $

OCE Lbs 16705 5846 27545 4176 7534 1800

OCE $ 6479.44 1219.21 5222.25 928.32 1722.05 1061.97

1 2 3 4 5 60

20000

40000

60000

80000

100000

120000

Pre Training

Post Training

Pre Training 115390 25086.58

Post Training 63606 16633.24

OCE Lbs OCE $

Machines

0

10000

20000

30000

40000

50000

60000

Pre Training

Post Training

Pre Training 53124 7990.54

Post Training 11684 2544.99

OCE Lbs OCE $0

2000

4000

6000

8000

10000

12000

14000

16000

Pre Training

Post Training

Pre Training 8505 2442.17

Post Training 14778 4192.15

OCE Lbs OCE $

Teams w/ Improved Metrics

• Machine 1 Team D

• Machine 2 Teams A, B, C

• Machine 3 Teams A, B, C, D

• Machine 4 Teams A, B, C, D

• Machine 5 Teams C, D

Survey

• Sent to members of the training team

• Questions– What factors contributed most to success of

training intervention?

– How would you rank them?

Survey Results

The respondents to the survey ranked these factors in the following order.

2. Support of upper management

3. Effectiveness of the Training Coordinators

4. Structured training and certification process

5. Quality of the training materials

Survey Results (Cont.)

5. (tie) Location of training center Diversity of training materials Focus on safety

6. Standardization of training and qualifications

7. Hourly Workforce Training Survey and Needs Assessment utilized to determine top training needs

8. Effect on employee morale

Summary

• Management support

• Needs analysis

• Specific business goals• Stakeholder involvement

• Isolate the effects of training

• Use appropriate metrics

• Measure appropriate units