Validity of Observational Job Analysis Methods Brian D. Lowe, Ph.D., CPE National Institute for Occupational Safety and Health Cincinnati, OH August 12,

Validity of Observational Validity of Observational Job Analysis MethodsJob Analysis Methods

Brian D. Lowe, Ph.D., CPEBrian D. Lowe, Ph.D., CPENational Institute for Occupational Safety and HealthNational Institute for Occupational Safety and Health

Cincinnati, OHCincinnati, OH

August 12, 2003August 12, 2003

presentation outline

Physical risk factors for WMSDs and job analysis methods for their characterization

NIOSH study of observational job analysis methods

Methods Results Conclusions

Validity considerations in job analysis

methods for assessing WMSD risk factorsmethods for assessing WMSD risk factors

Job Titles/SIC codeJob Titles/SIC code

Worker Self ReportWorker Self Report

Systematic ObservationSystematic Observation

Direct Measurement Direct Measurement (Instrumentation)(Instrumentation)

increasingreliability &precision

increasingconvenience

goals for exposure characterization(Kilbom, 1994)

• External Validity - identify exposures associated with increased risk for WMSDs

epidemiology

• Internal Validity - exposure is classified accurately relative to a known standard

biomechanics

Exposure Response

ObjectiveObjective

Group methods of scaling risk factors used in Group methods of scaling risk factors used in observational-based job analysesobservational-based job analyses

Compare observational estimates of risk factors Compare observational estimates of risk factors with instrumentation-based measureswith instrumentation-based measures electrogoniometer – wrist/forearm posture/kinematicselectrogoniometer – wrist/forearm posture/kinematics

optical motion capture – shoulder posture/kinematicsoptical motion capture – shoulder posture/kinematics

electromyography – force of exertionelectromyography – force of exertion

explore the likelihood and nature of errors in explore the likelihood and nature of errors in exposure characterizationexposure characterization

jobs simulated in the laboratoryjobs simulated in the laboratory

Job A~ 13 s

Job B ~ 8 s

Job C~ 56 s

Job D~ 46 s

electrogoniometerelectrogoniometer

-120

-80

-40

0

40

80

120

0 10 20 30 40 50 60

flexion/extension (α) supination/pronationJob C - cycle 3

angl

e (d

eg)

α

optical motion captureoptical motion capture

A: 4 – 2 HCSxB: 4 – 3C: A x B HCSyD: A x C HCSzE: 7 – 5 TCSZF: 7 – 6G: F x E TCSYH: G x E TCSX

ZT

-YT

-XT

-zH

-yH

xH

5

6

7

4

3

2

A

E

F

B

trunk

humerus4

2

3anatomicalpod

A: 4 – 2 HCSxB: 4 – 3C: A x B HCSyD: A x C HCSzE: 7 – 5 TCSZF: 7 – 6G: F x E TCSYH: G x E TCSX

ZT

-YT

-XT

-zH

-yH

xH

5

6

7

4

3

2

A

E

F

B

trunk

humerus4

2

3

4

2

3

2

3anatomicalpod

motion capture – shoulder kinematicsmotion capture – shoulder kinematics = cos-1 (X · x) = cos-1 [(Y · x)/sin()] = cos-1 [ -(X · y)/sin()]

x – z’ – x” Euler angle sequence: Rotation about x : Rotation about z’: Rotation about x”

- shoulder elevation - plane of shoulder elevation

0

0

30

60

90

120

0 10 20 30 40 50

elbow

sh elev

pl sh elev

-90

-60

-30

0

30

60

90

120

0 10 20 30 40 50

flex/ext

pro/sup

14.8000 s 14.8333 s14.7667 s

join

t a

ngl

e (d

egr

ees

)jo

int

an

gle

(de

gre

es)

time (s)

cycle 1 cycle 2 cycle 3 cycle 4

0

30

60

90

120

0 10 20 30 40 50

elbow

sh elev

pl sh elev

-90

-60

-30

0

30

60

90

120

0 10 20 30 40 50

flex/ext

pro/sup

14.8000 s 14.8333 s14.7667 s

join

t a

ngl

e (d

egr

ees

)jo

int

an

gle

(de

gre

es)

time (s)

cycle 1 cycle 2 cycle 3 cycle 4

video and instrumentation synchronizationvideo and instrumentation synchronization

participants and procedureparticipants and procedure

ParticipantsParticipants

28 professional ergonomists28 professional ergonomists

14 from academia,14 from industry/consulting14 from academia,14 from industry/consulting

12 - Ph.D./M.D., 13 - M.S., 3 - B.S.12 - Ph.D./M.D., 13 - M.S., 3 - B.S.

Years experience in ergonomics (1 – 30 yrs.)Years experience in ergonomics (1 – 30 yrs.)

ProcedureProcedure

Assigned one method for posture analysisAssigned one method for posture analysis

Estimated posture from video recording of jobs Estimated posture from video recording of jobs

Analyses were unguidedAnalyses were unguided

posture scalingposture scalingmethod 1 – 3 categoriesmethod 1 – 3 categories

11 22 33elbow flex elbow flex (deg)(deg) <40<40°° 4040°°-80-80°° >80>80°°

shoulder elev shoulder elev (deg)(deg) 00°°-40-40°° 4040°°-80-80°° >80>80°°

plane of sh elev plane of sh elev (deg)(deg) <30<30°° 3030°°-90-90°° >90>90°°

11 22 33

modemodepeakpeak wrist flex wrist flex (deg)(deg) >20>20°° 2020°°-0-0°°

peakpeak wrist ext wrist ext (deg)(deg) 00°°-20-20°° >20>20°°

modemodepeakpeak forearm sup forearm sup (deg)(deg) >40>40°° 4040°°-0-0°°

peakpeak forearm pro forearm pro (deg)(deg) 00°°-40-40°° >40>40°°

posture scalingposture scalingmethod 2 – 6 categoriesmethod 2 – 6 categories

11 22 33 44 55 66wrist flexwrist flex >45>45 45-20 45-20 20 -020 -0

wrist extwrist ext 0-200-20 20-4520-45 >45>45

forearm supforearm sup >60>60 60-3060-30 30-030-0

forearm proforearm pro 0-300-30 30-6030-60 >60>60

elbow flexelbow flex <20<20 20-4020-40 40-6040-60 60-8060-80 80-10080-100 >100>100

shoulder elevshoulder elev <20<20 20-4020-40 40-6040-60 60-8060-80 80-10080-100 >100>100

plane of sh elevplane of sh elev <0<0 0-300-30 30-6030-60 60-9060-90 90-12090-120 >120>120

posture scalingposture scalingmethod 3 - visual analog scale (VAS)method 3 - visual analog scale (VAS)

0neutral

10extreme

2 4 6 8

wrist flexionwrist extension

forearm supinationforearm pronation

elbow flexionshoulder elevation

plane of shoulder elevation

0°0°0°0° 0°0°0°

95°85°

145°135°150°180°150°

ResultsResultswrist/forearm – 3 categories (method 1)wrist/forearm – 3 categories (method 1)

0%

20%

40%

60%

80%

100%

-2 -1 0 1 2

error (# of categories)

prob

abilit

y

mode flex/ext

peak flex

peak ext

0%

20%

40%

60%

80%

100%

-2 -1 0 1 2error (# of categories)

pro

ba

bili

ty

mode sup/pro

peak sup

peak pro

error = estimated - measured

elbow/shoulder – 3 categories elbow/shoulder – 3 categories (method 1)(method 1)

mode

0%

20%

40%

60%

80%

100%

-2 -1 0 1 2error (# of categories)

prob

abili

ty

elbow flex

sh elev

pl sh elev

peak

0%

20%

40%

60%

80%

100%

-2 -1 0 1 2error (# of categories)

prob

abili

ty

elbow flex

sh elev

pl sh elev

wrist/forearm – 6 categories wrist/forearm – 6 categories (method 2)(method 2)

0%

10%

20%

30%

40%

50%

60%

-5 -4 -3 -2 -1 0 1 2 3 4 5

error (# of categories)

pro

ba

bili

ty

mode flex/ext

peak flex

peak ext

0%

10%

20%

30%

40%

50%

60%

-5 -4 -3 -2 -1 0 1 2 3 4 5error (#of categories)

pro

ba

bili

ty

mode sup/propeak suppeak pro

elbow/shoulder – 6 categories elbow/shoulder – 6 categories (method 2)(method 2)mode

0%

20%

40%

60%

-4 -3 -2 -1 0 1 2 3 4error (# of categories)

prob

abili

ty

elbow flexsh elevpl sh elev

peak

0%

20%

40%

60%

-4 -3 -2 -1 0 1 2 3 4

error (# of categories)

prob

abili

ty

elbow flex

sh elevpl sh elev

VAS – flexion/extension VAS – flexion/extension (method 3)(method 3)

peak average

wrist flexion wrist extension

r2 = 0.31*r2 = 0.28*

r2 = 0.02 r2 = 0.00

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

measured

est

ima

ted

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

measured

est

ima

ted

VAS – supination/pronation VAS – supination/pronation (method 3)(method 3)

peak average

forearm supination forearm pronation

r2 = 0.02r2 = 0.03

r2 = 0.02 r2 = 0.09

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

measured

est

ima

ted

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

measurede

stim

ate

d

VAS – shoulder and elbow VAS – shoulder and elbow (method 3)(method 3)

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

measured

estim

ated

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

measured

estim

ated

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

measured

estim

ated peak average

elbow flexion shoulder elevation plane of shoulder elev +

r2 = 0.47*r2 = 0.49*

r2 = 0.66*r2 = 0.46*

r2 = 0.03r2 = 0.18*

temporal distribution of posture temporal distribution of posture (wrist/forearm – 3 category)(wrist/forearm – 3 category)

0%

20%

40%

60%

80%

>20º f lex neutral(±20º)

>20º ext >40º sup neutral(±40º)

>40º pro

w rist f lex/ext forearm sup/pro

estimatedmeasured

per

cent

of w

ork

cyc

le

N N

N = neutral posture

temporal distribution of posturetemporal distribution of posture(wrist/forearm – 6 category)(wrist/forearm – 6 category)

0%

20%

40%

60%>

45°

f

45°

- 20

° f

20°

- 0°

f

0° -

20°

e

20°

- 45

° e

> 45

° e

> 60

° s

60°

- 30

° s

30°

- 0°

s

0° -

30°

p

30°

- 60

° p

> 60

° p

w rist f lex/ext forearm sup/pro

estimatedmeasured

per

cent

of w

ork

cyc

le

N N

temporal distribution of posturetemporal distribution of posture(elbow/shoulder – 3 category)(elbow/shoulder – 3 category)

0%

20%

40%

60%

80%

100%

0º - 40º 40º - 80º > 80º 0º - 40º 40º - 80º > 80º < 30º 30º - 90º > 90º

elbow flex/ext shoulder elev shoulder plane of elev

estimatedmeasured

per

cent

of w

ork

cyc

le

N N N

temporal distribution of posturetemporal distribution of posture (elbow/shoulder – 6 category)(elbow/shoulder – 6 category)

0%

20%

40%

60%0º

-20º

20º-

40º

40º-

60º

60º-

80º

80º-

100º

>100

º

0º-2

0º

20º-

40º

40º-

60º

60º-

80º

80º-

100º

>100

º

<0º

0º-3

0º

30º-

60º

60º-

90º

90º-

120º

>120

º

elbow flex/ext shoulder elev shoulder plane elev

estimatedmeasured

per

cent

of w

ork

cyc

le

N N N

DiscussionDiscussion

Performance does not necessarily reflect Performance does not necessarily reflect bestbest casecaseLimitations of the StudyLimitations of the Study Single video viewSingle video view Simulated job tasks (laboratory study)Simulated job tasks (laboratory study) Analysts had no familiarity with jobsAnalysts had no familiarity with jobs Methods may not have been familiar to analystsMethods may not have been familiar to analysts Little information regarding the strategy analysts usedLittle information regarding the strategy analysts used

Intended to reflect performance in the Intended to reflect performance in the typicaltypical case case

summary of findingssummary of findings

Posture classification accuracy related to the Posture classification accuracy related to the size of the joint/limb segments (Genaidy size of the joint/limb segments (Genaidy et alet al, , 1993; Baluyut 1993; Baluyut et alet al, 1995), 1995)

Posture classification accuracy related to the Posture classification accuracy related to the number of scale categoriesnumber of scale categories

p(correct classification) = 73% for most frequent p(correct classification) = 73% for most frequent shoulder/elbow posture w/3 categoriesshoulder/elbow posture w/3 categories

p(correct classification) = 30% for most frequent p(correct classification) = 30% for most frequent wrist/forearm posture w/6 categorieswrist/forearm posture w/6 categories

validity considerations in job analysisvalidity considerations in job analysis

Misclassification of working posture occurred Misclassification of working posture occurred in job analyses even when using a small in job analyses even when using a small number of posture categoriesnumber of posture categories

Posture misclassifications with higher Posture misclassifications with higher precision scale were more frequent, but their precision scale were more frequent, but their effect is lesseffect is less

Duration severity of posture tended to be Duration severity of posture tended to be underestimatedunderestimated

Disclaimer

Mention of any company name or product, or inclusion of any reference, does not constitute endorsement by the National Institute for Occupational Safety and Health.

Acknowledgment

The contributions of Dan Habes, NIOSH, Ed Krieg, NIOSH, and Ahmed Khalil, University of Cincinnati are greatly appreciated.

risk factors in physical workrisk factors in physical work

risk factors for work related musculoskeletal risk factors for work related musculoskeletal disorders (WMSDs)disorders (WMSDs)

postureposture

forceforce

repetitionrepetition

vibrationvibration

Ergonomic Exposure Assessment – Ergonomic Exposure Assessment – Observational AccuracyObservational Accuracy

temporal scalingmag

nitu

de s

cali

ng

time

post

ure

accuracy

lab simulation video recording presented to ergonomists

Motion AnalysisGoniometer

observation

job analysis methods for the systematic job analysis methods for the systematic observation of postureobservation of posture

peak (peak posture)

average (average posture)

mode (category with most frequently

occurring posture)

work sampling (periodic

observations)

near continuous (posture category vs.

time recording)

low (2 - 3 categories)

medium (4 - 6 categories)

high (> 6 categories, VAS

method or continuous

scale)

increasing difficulty

RULA

STRAIN INDEX

Keyserling (1986)

Armstrong et al (1982)

OCRA

Latko (1997)

OWAS

Drury (1987)

Temporal

Sp

atia

l

work cycle analysiswork cycle analysis

shoulder elevation – Job C

0

40

80

120

0 20 40 60 80 100 120 140 160 180 200 220

time (s)

angu

lar

posi

tion

(deg

)

cycle 1 cycle 2 cycle 3 cycle 4

upper limb postures evaluatedupper limb postures evaluatedelectrogoniometer

optical motion

capture

summary of other findingssummary of other findings

Time to completion of the analysis was not Time to completion of the analysis was not related to the resulting accuracyrelated to the resulting accuracy

No relationship between years experience and No relationship between years experience and accuracy of observational estimatesaccuracy of observational estimates

No relationship between work cycle variability No relationship between work cycle variability and accuracy of observational estimatesand accuracy of observational estimates

radial/ulnar deviationradial/ulnar deviation

Inter-rater agreement statisticsInter-rater agreement statistics

Intraclass correlation coefficient among raters Intraclass correlation coefficient among raters (ergonomists) less than for flex/ext, sup/pro(ergonomists) less than for flex/ext, sup/pro

3-category3-category 6-category6-category

flex/extflex/ext 0.2290.229 0.3420.342

pro/suppro/sup 0.2150.215 0.3080.308

rad/ulnrad/uln 0.2170.217 0.1230.123

Juul-Kristensen et al. (1997)Juul-Kristensen et al. (1997)

0%

20%

40%

60%

80%

flexion mild flexion neutral mild extension extension

pe

rce

nt o

f wo

rk c

ycle

analysts - present study

measured - present study

analyst - Juul Kristensen et al.

measured - Juul Kristensen et al.

Electrogoniometer Calibration

RR22 maximum errormaximum error

flex/extflex/ext 0.990.99 0.50.5°° @ 45 @ 45°° flex flex

sup/prosup/pro 0.940.94 2.52.5°° @ 45 @ 45°° pro pro

rad/ulnrad/uln 0.800.80 1010°° @ 30 @ 30°° uln uln

choice of ROM as VAS anchorchoice of ROM as VAS anchor

0° 100°

0° 80°

true magnitude

75%

60%

60°

Observation vs. ChanceObservation vs. Chance

estimated

> 60 sup 30 - 60 sup 0 - 30 sup 0 - 30 pro 30 - 60 pro > 60 pro marginal

> 60 sup 0.000 0.000 0.000 0.000 0.000 0.000 0.000

30 - 60 sup 0.000 0.000 0.000 0.000 0.000 0.000 0.000

measured 0 - 30 sup 0.000 0.000 0.025 0.075 0.000 0.000 0.100

0 - 30 pro 0.000 0.000 0.025 0.150 0.075 0.050 0.300

30 - 60 pro 0.000 0.000 0.000 0.050 0.000 0.025 0.075

> 60 pro 0.000 0.075 0.075 0.175 0.000 0.200 0.525

marginal 0.000 0.075 0.125 0.450 0.075 0.275 1.000

> 60 sup 30 - 60 sup 0 - 30 sup 0 - 30 pro 30 - 60 pro > 60 pro marginal

> 60 sup 0.000 0.000 0.000 0.000 0.000 0.000 0.000

30 - 60 sup 0.000 0.000 0.000 0.000 0.000 0.000 0.000

measured 0 - 30 sup 0.017 0.017 0.017 0.017 0.017 0.017 0.100

0 - 30 pro 0.050 0.050 0.050 0.050 0.050 0.050 0.300

30 - 60 pro 0.013 0.013 0.013 0.013 0.013 0.013 0.075

> 60 pro 0.088 0.088 0.088 0.088 0.088 0.088 0.525

marginal 0.167 0.167 0.167 0.167 0.167 0.167 1.000

ergonomists’observation

chance