A Review of EAP Evaluation in the 1990s

PLEASE SCROLL DOWN FOR ARTICLE

This article was downloaded by: [Canadian Research Knowledge Network]On: 30 July 2010Access details: Access Details: [subscription number 783016864]Publisher RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Employee Assistance QuarterlyPublication details, including instructions for authors and subscription information:http://www.informaworld.com/smpp/title~content=t904092371

A Review of EAP Evaluation in the 1990sRick Csiernika

a School of Social Work, King's University College at the University of Western Ontario,

To cite this Article Csiernik, Rick(2005) 'A Review of EAP Evaluation in the 1990s', Employee Assistance Quarterly, 19: 4,21 — 37To link to this Article: DOI: 10.1300/J022v19n04_02URL: http://dx.doi.org/10.1300/J022v19n04_02

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf

This article may be used for research, teaching and private study purposes. Any substantial orsystematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply ordistribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contentswill be complete or accurate or up to date. The accuracy of any instructions, formulae and drug dosesshould be independently verified with primary sources. The publisher shall not be liable for any loss,actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directlyor indirectly in connection with or arising out of the use of this material.

http://www.informaworld.com/smpp/title~content=t904092371

http://dx.doi.org/10.1300/J022v19n04_02

http://www.informaworld.com/terms-and-conditions-of-access.pdf

A Review of EAP Evaluation in the 1990s

Rick Csiernik

ABSTRACT. A review of the literature found 39 published EAP evalu-ations during the 1990s: four examining needs assessments, 12 casestudies, nine cost-benefit analyses, five on client satisfaction along withnine process evaluations. The majority of evaluations were produced byorganizations with internal EAPs located in the United States thoughthere were 12 conducted by organizations employing external serviceproviders and also a few from other nations, notably Canada, the UnitedKingdom and Australia, found in the review. A variety of methodologiesfrom key informant interviews to formal controlled experiments wereused in the 39 evaluations employing both quantitative and qualitativedata collection methods and from one to five distinct data sources. Whilethis indicates a growing robustness in EAP evaluation and the ongoingawareness of the importance of conducting and publishing evaluations,it also underscores the lack of uniformity in systematically examiningthis dynamic practice area. [Article copies available for a fee from TheHaworth Document Delivery Service: 1-800-HAWORTH. E-mail address:<[email protected]> Website: <http://www.HaworthPress.com> 2004 by The Haworth Press, Inc. All rights reserved.]

KEYWORDS. Needs assessment, case study, cost-benefit, client satis-faction, process evaluation

Rick Csiernik, PhD, is Associate Professor and Graduate Studies Coordinator at theSchool of Social Work, King’s University College at the University of Western Ontario.

Address correspondence to: Rick Csiernik, PhD, 125 East 8th Street, Hamilton, On-tario, Canada L9A 4Y7 (E-mail: [email protected] or [email protected]).

Employee Assistance Quarterly, Vol. 19(4) 2004Available online at http://www.haworthpress.com/web/EAQ

2004 by The Haworth Press, Inc. All rights reserved.Digital Object Identifier: 10.1300/J022v19n04_02 21

Downloaded By: [Canadian Research Knowledge Network] At: 02:53 30 July 2010

INTRODUCTION

The Employee Assistance field is a multidisciplinary, multi-perspec-tive, international arena of practice and while it remains a relativelysmall field it nonetheless attracts a myriad of standpoints on how pro-gramming should be best instituted. For the past two decades authorshave attempted to bring together the limited though important researchand evaluation findings from programs in collections to better informand enlighten practitioners, EAP committee members and human re-source leaders (Csiernik, 1995; 1998; Jerrell & Righmyer, 1982; Kurtz,Googins & Howard, 1984). This article follows in that tradition and re-mains pertinent as the EAP field continues to struggle in defining whatexactly is best practice (see Csiernik, 2003a; 2003b). Using Macdon-ald’s (1986) five steps in a comprehensive evaluation (Table 1) resultsfrom worksite evaluations published in the 1990s are presented forinformation and to examine trends and outcomes in the field.

NEEDS ASSESSMENT

An EAP should have, both at the time the program is established andon a periodic basis thereafter, a mechanism to assess the needs for em-ployee assistance. The EAP needs assessment should be aimed at iden-tifying worksite environmental, programmatic, personnel and stress-related factors deleterious to employee well-being or productivity. Pro-gram decisions should be directly related to the assessment findings andbe periodically evaluated in that context (U.S. Department of Healthand Human Services, 1986: 10).

During the 1990s four needs assessments pertaining to Employee As-sistance Programming (EAP) were published (Table 2). Two of these

22 EMPLOYEE ASSISTANCE QUARTERLY

TABLE 1. Five Steps in a Comprehensive Evaluation

1. Needs Assessment – to determine overall program goals and direction

2. Program Development (case studies) – to describe the program, its rationale and objectives

3. Input Evaluation – to determine if the program components have been correctly implemented

4. Outcome Evaluation – to determine if the program objectives have been achieved

5. Process Evaluation – to determine what the program is actually doing and how well


TA

BLE

2.N

eeds

Ass

essm

ents

Aut

hors

Ber

man

,Sul

sky,

Par

gam

ent,

Don

ovan

,Kur

zman

&R

odrig

uez

&S

afye

r,Li

tchf

ield

&B

alze

r&

Kau

sch

Rot

man

Bor

gen

Leah

yY

ear

1991

1993

1998

1995

Wor

kpla

ceN

orth

wes

tOhi

oN

ewY

ork

City

Wes

tern

Can

ada

Nor

thea

stU

nite

dS

tate

sM

unic

ipal

Gov

ernm

ent

Hom

eC

are

Wor

kers

Var

iety

Hig

hT

echn

olog

yC

ompa

nyW

orkf

orce

Siz

e22

5N

otre

port

ed44

,000

6,26

0S

ampl

eS

ize

215

404

rand

omly

sele

cted

90E

AP

adm

inis

trat

ors

6,26

0vo

lunt

eers

Res

earc

hM

etho

dsE

mpl

oyee

reco

rds

revi

ew

Key

info

rman

tint

ervi

ews

Sel

f-ad

min

iste

red

Indi

vidu

alin

terv

iew

sS

elf-

adm

inis

tere

dS

elf-

adm

inis

tere

dqu

estio

nnai

re–2

0sc

ales

ques

tionn

aire

ques

tionn

aire

one

hour

inle

ngth

2ho

urs

inle

ngth

10se

ctio

ns54

%re

spon

sera

te20

%re

spon

sera

te69

%re

spon

sera

te47

%re

spon

sera

te

Out

com

eLe

dto

sele

ctio

nof

Soc

ialw

orke

rhi

red

befo

reP

erso

nalp

robl

ems

Foc

used

onw

hata

nex

tern

alse

rvic

ene

eds

asse

ssm

entf

ully

com

mon

atw

ork

but

exis

iting

EA

Pco

uld

dode

liver

ym

odel

and

com

plet

edin

resp

onse

toE

AP

sst

illun

der-

toas

sist

wor

king

pare

nts

mec

hani

sms

tops

ycho

soci

alne

eds

ofut

ilize

dby

wor

kfor

cew

ithad

oles

cent

sfa

cilia

tere

ferr

als

resp

onde

nts

Pro

vide

dre

com

men

datio

nsM

embe

ras

sist

ance

plan

Add

ition

altr

aini

ngan

dfo

rpr

oact

ive

prog

amin

stitu

edup

onco

mpl

etio

npr

omot

ion

war

rant

edre

spon

seof

rese

arch

23


were used to assist in the development of new programs, while two oth-ers were conducted to examine issues related to more mature and estab-lished programs. In three of the needs assessments self-administeredquestionnaires were the primary mechanism of data collection with oneassessment employing face-to-face individual interviews and anothersupplementing the questionnaire with key informant interviews and areview of employee records. Response rates ranged from 20% to 69%though in each instance the needs assessment was used to direct pro-gram development. The two assessments of new programs led to the de-velopment of quite different looking EAPs, one an external model, thesecond a peer-based initiative, reflective of the two distinct workplaceenvironments and the value of instituting a needs assessment. Bothneeds assessments of mature programs were able to highlight areas ofrequired program development to assist the two EAP committees in fu-ture direction and which highlight the U.S. Department of Health andHuman Services best practice recommendation that needs assessmentbe used for periodic EAP evaluative review.

CASE STUDIES

The second step in the comprehensive evaluation process recom-mended by Macdonald is to describe a program, its rational and its ob-jectives, once it is functional. Case studies are exploratory yet empirical,descriptive and detailed studies using a range of research methods to re-port on and interpret a single example of a phenomenon (Merriam,1988; Tesch, 1990; Yin, 1989). This type of evaluation has historicallybeen the most popular and prominent mechanism through which thevarious forms of EAP have been discussed and described (Csiernik,1995; 1997).

During the 1990s there were proportionately fewer case studiesfound in the literature (Table 3) than have typically been publishedthough all that were did provide varying degrees of insight and narrativeinto the historic development of the individual programs. Eight of thetwelve case studies used internal service provision models, two ofwhich had a peer support orientation. Both of the peer-based programswere statewide initiatives and both were from the health sector, one fornurses in Indiana, the other for physicians in Colorado. As has been typ-ical in the literature, a minority of the case studies published (n = 3) re-lated to EAP whose assistance was provided by an external service, oneof which was a consortium design. The final case study discussed an in-



TA

BLE

3.C

ase

Stu

dies

Sum

mar

y

Aut

hor

Yea

rW

orkp

lace

Wor

kfor

ceP

rogr

amE

AP

His

tory

Pro

blem

Ref

erra

lsP

olic

esan

dC

ase

Siz

eIn

itiat

edD

eliv

ery

Out

lined

Pro

file

Pro

cedu

res

His

tory

Car

lson

etal

.19

94E

pisc

opal

Chu

rch,

Roc

hest

erN

.Y.

500

1988

exte

rnal

yes

yes

yes

Coo

per

etal

.19

90P

osta

lSer

vice

,Nor

thE

ngla

nd19

86in

tern

alye

sye

sye

sD

ilts

etal

.19

94C

olor

ado

Phy

sici

ans

8,50

019

86pe

erye

sye

sye

sye

sE

very

&Le

ong

1994

San

Ono

fre

Nuc

lear

Gen

erat

ing

Sta

tion

2,47

019

89in

tern

alye

sye

sF

inke

etal

.19

96In

dian

aS

tate

Nur

ses’

Ass

ocia

tion

1984

peer

yes

yes

yes

Fly

nnet

al.

1993

Uni

ted

Air

Line

s–P

ilotA

ssis

tanc

e19

73in

tern

alye

sye

sK

elly

1995

Con

greg

atio

nalA

ssis

tanc

e15

019

90as

sess

men

tye

sye

sP

rogr

am,I

llino

isch

urch

esan

dre

fer

Kur

tzet

al.

1996

Pos

talS

ervi

ce,U

nite

dS

tate

s19

68ex

tern

alye

sye

sM

aide

net

al.

1993

Mul

tiple

site

s,N

orth

Cen

tral

Illin

ois

1,76

219

89co

nsor

tium

yes

Sch

efst

adet

al.

1999

Hor

seca

reW

orke

rs,M

aryl

and

1989

inte

rnal

yes

yes

yes

Sto

er-S

cagg

s19

90O

hio

Sta

teU

nive

rsity

,Col

umbu

s,O

hio

1986

inte

rnal

yes

Whi

teet

al.

1996

Uni

vers

ityof

Mar

ylan

dM

edic

al6,

400

1989

inte

rnal

yes

yes

Sys

tem

–Bal

timor

e,M

aryl

and

25


novative assessment and referral program developed for use with thecongregations of 150 churches in Illinois.

Seven of these evaluations provided problem profiles presented byemployees while five discussed referral routes to the program, two poli-cies and procedures and two provided detailed case histories of clientswho had used the particular EAP and the outcome of that use. Also ofinterest to note was the time of program initiation compared to the timeof article publication. Only two, the United States Postal Service EAPand the United Air Lines Pilot Assistance Program, were well estab-lished. The average time between initiation and article publication forthe other ten case studies was 6.5 years, with a range of four to ten,enough time to watch a fledgling initiative take hold and begin to havesignificant effects on the lives of individuals and on the organizationitself. However, this time frame is also sufficient to allow for more ex-tensive evaluation to supplement the primarily narrative case studypresentation.

INPUT EVALUATION

No input evaluations were found in the literature. This may be ex-plained by a variety of factors including:

• the rudimentary nature of this type of evaluation;• the lack of perceived value in the findings;• that input evaluations are simply not routinely conducted; and/or,• that the findings are not formally documented.

While this form of evaluation may not be or may not be perceived tobe of significance by program evaluators and authors it can still be ofvalue for an EAP committee and an organization to conduct an inputevaluation to examine what the program was intended to do and if it ismeeting those objectives.

OUTCOME EVALUATIONS

An EAP should have a mechanism in place to evaluate the appropri-ateness, effectiveness, and efficiency of the delivery of services andprogram integration. Evaluations of the scope and appropriateness ofclient services, educational programs, supervisory training, and out-



reach activities should be performed on an annual basis and become partof the permanent program records (U.S. Department of Health andHuman Services, 1986: 16).

The most common form of outcome evaluation in the EAP sector hasbeen the traditional cost-benefit study. However, during the 1990s therewere few formal studies presented. More typical were briefer outcomereports that did not provide a broader literature review, context for thestudy or discussion of the organization but rather focused instead pri-marily on methodology and outcome (Table 4). The one exception tothis was Blaze-Temple and Howatt’s (1997) examination of an Austra-lian EAP; however, this was part of a larger evaluation that was pre-sented as two articles and in effect the cost-benefit study was part of amore detailed process evaluation (see Blaze-Temple & Honig, 1997).

As is the trend in EAP evaluation the majority of the reports wereconducted by organizations with internal programs. Of the six programsthat reported the size of their workforce only one employed fewer thanfive thousand employees. Thus, larger organizations were overrepre-sented in this type of evaluation study. Variables examined to assesscost-benefit ranged from one, medical claims, at Southern CaliforniaEdison (Conlin, Amaral & Harlow, 1996) to five at Burlington Northern(Schear, 1995) with a mean of 2.8 with no one variable being used in alleight reports. The most common costs studied in the eight reports weremedical costs (5), retention/termination (4), absenteeism (4) and sicktime (3). The length of study ranged from one year at Burlington North-ern (Schear, 1995) to an examination of retention rates over ten years atChevron (Collins, 1998). Despite the brevity of the published reportsthe study periods themselves were of good duration averaging nearlyfive years in length.

As expected each study demonstrated the positive financial conse-quences of establishing and maintaining an EAP. Cost savings were re-ported at $65,431 for one year at the University of Michigan MedicalCenter (Bruhnsen, 1994), $7.21 returned for every dollar spent by theUnited States Postal Service on their EAP (Stephenson & Bingaman,1999) and $14.00 at Chevron (Collins, 1998) to a future anticipated off-set of six million dollars at McDonnell Douglas (Editorial Board, 1993).Job performance ratings were indicated to improve (Schear, 1995),medical costs drop (Conlin, Amaral & Harlow, 1996; Yandrick 1992a;1992b) and absenteeism become less of a problem (Bruhnsen, 1994,Editorial Board, 1993; Stephenson & Bingaman, 1999; Yandrick, 1992a)post-EAP use.

Rick Csiernik 27


TA

BLE

4.C

ost-

Ben

efit

Rep

orts

Aut

hor

Bla

ze-T

empl

e&

How

att

Bru

hnse

nC

ollin

sC

onlin

,Am

aral

&H

arlo

wY

ear

1997

1994

1998

1996

Wor

kpla

ceH

ospi

talL

aund

ryan

dLi

nen

Uni

.ofM

ichi

gan

Med

ical

Che

vron

Cor

pora

tion

Sou

ther

nC

alifo

rnia

Per

th,A

ustr

alia

Cen

ter,

Ann

Arb

orE

diso

nW

orkf

orce

Siz

e40

08,

000

EA

PD

eliv

ery

Ext

erna

lIn

tern

alIn

tern

alIn

tern

al

Met

hod

Qua

si-e

xper

imen

tal

Mat

ched

coho

rtof

122

Rev

iew

ofal

lEA

Pfil

esM

atch

edco

hort

ofm

ultip

letim

ese

ries

form

erE

AP

clie

nts

from

1990

-199

4em

ploy

ees

with

subs

tanc

e2

pre-

3po

stE

AP

use

n=

11,7

73ab

use

clai

ms

EA

Pco

unse

lling

vs.

Gro

ups

mat

ched

byag

e,S

afet

yra

tes:

prog

ram

30em

ploy

ees

EA

Pus

ese

lf-ar

rang

edvs

.se

x,ed

ucat

ion,

job

user

svs

non-

user

s29

noE

AP

cont

act-

only

noco

unse

lling

clas

san

dye

ars

ofse

rvic

eR

eten

tion

rate

s:19

92m

anag

edca

reus

eC

AT

OR

stud

yba

selin

e

Stu

dyT

ime

Fra

me

2.5

year

s5

year

sR

eten

tion

rate

s:10

year

s30

mon

ths

Pro

duct

ivity

:4ye

ars

Acc

iden

trat

es:3

year

sV

aria

bles

Exa

min

edA

bsen

teei

smR

eten

tion

Acc

iden

tsM

edic

alco

sts

Com

pens

atio

ncl

aim

sS

ick

time

Pro

duct

ivity

Sic

ktim

eR

eten

tion

Ret

entio

n

Out

com

eS

elf-

arra

nged

coun

selli

ngE

AP

clie

nts

had

ahi

gher

Cos

tret

urn

of14

:1E

AP

user

sha

dhi

gher

prod

uced

the

mos

tcos

tre

tent

ion

rate

and

noF

ewsu

bsta

nce

abus

ing

initi

alad

dict

ion

trea

tmen

tsa

ving

sas

ther

eis

nodi

ffere

nce

insi

ckda

ysem

ploy

ees

fired

cost

sbu

tsub

eque

ntly

prog

ram

cost

toth

ele

adin

gto

aco

stIm

prov

emen

tsin

both

low

erm

enta

land

phys

ical

empl

oyer

savi

ngs

of$6

5,34

1su

perv

isor

yre

ferr

als

heal

thca

reco

sts

(mea

n50

%)

and

self-

EA

Pus

ers

had

38%

EA

Pw

asco

stne

utra

lre

ferr

ed(m

ean

5%)

clie

nts

less

med

ical

cost

sov

erS

afet

yra

tes

iden

tical

30-m

onth

post

-tre

atm

ent

perio

d

28


Aut

hor

Edi

toria

lBoa

rdS

chea

rS

teph

enso

n&

Bin

gam

anY

andr

ick

Yan

dric

kY

ear

1993

1995

1999

1992

a19

92b

Wor

kpla

ceM

cDon

nell

Dou

glas

Bur

lingt

on-N

orth

ern

Uni

ted

Sta

tes

Ora

nge

Cou

nty

Flo

rida

Virg

inia

Pow

erU

nite

dS

tate

sP

osta

lSer

vice

Pub

licS

choo

lsR

ichm

ond,

Virg

inia

Wor

kfor

ceS

ize

130,

000

900,

000

19,0

0012

,000

EA

PD

eliv

ery

Inte

rnal

Inte

rnal

Inte

rnal

Inte

rnal

Met

hod

Com

paris

onof

EA

Pus

ers

Com

paris

onof

cost

sP

rosp

ectiv

eco

st-b

enef

itC

ompa

rison

ofhe

alth

Pre

-pos

tEA

Pto

thos

ew

ithpr

oble

ms

the

mon

thpr

ior

toan

alys

isus

ing

first

time

cost

sin

one

year

usin

gus

eco

mpa

rison

who

soug

hthe

lpou

tsid

eE

AP

use

toon

eye

arE

AP

user

s5

sets

of25

empl

oyee

sto

thos

eus

ing

the

EA

Pto

aco

ntro

lgro

upaf

ter

EA

Pus

etie

red

acro

ss4

year

sbe

havi

oral

heal

thof

thos

ew

ithou

tof

EA

Pus

eco

mpa

red

tobe

nefit

ssu

bsta

nce

abus

eof

25no

n-pr

ogra

mus

ers

men

talh

ealth

issu

es

Stu

dyT

ime

Fra

me

5ye

ars

1ye

ar5

year

pros

pect

ive

5ye

ars

8ye

ars

Var

iabl

esE

xam

ined

Abs

ente

eism

Abs

ente

eism

Abs

ente

eism

Med

ical

cost

sB

enef

itco

sts

Med

ical

cost

sA

ccid

ents

/inju

ries

Ben

efit

cost

sS

ick

time

Med

ical

cost

sT

erm

inat

ion

Med

ical

cost

sP

erfo

rman

ceC

osto

fEA

PW

orke

rs’c

ompe

nsat

ion

Out

com

eC

hem

ical

depe

nden

cyJo

bpe

rfor

man

cera

tings

Ben

efit

toco

stra

tio:

Bot

hm

edic

alco

sts

and

EA

Pus

ers

had

23%

clie

nts

had

29%

few

erby

supe

rvis

ors

incr

ease

dye

ar1:

1.27

:1cl

aim

sdr

opaf

ter

EA

Pus

elo

wer

med

ical

cost

sda

ysof

abse

nce,

42%

Few

erhe

alth

insu

ranc

eye

ar5:

7.21

:1an

dco

ntin

ueto

fall

over

and

15%

low

erfe

wer

term

inat

ions

,$7,

150

clai

ms,

med

ical

leav

esan

dtim

eno

n-be

havi

oral

less

inm

edic

alco

sts

abse

nces

.No

chan

ges

Cos

tsav

ings

incr

ease

heal

thbe

nefit

cost

sin

disa

bilit

yor

wor

kers

’po

stE

AP

use

Prio

rto

EA

Pus

ecl

ient

s’th

andi

dth

eM

enta

lhea

lthcl

ient

sco

mpe

nsat

ion

clai

ms

sick

leav

eab

ove

orga

niza

-co

mpr

ison

grou

pw

ere

abse

nt25

%le

ss,h

adtio

nav

erag

e,af

ter

EA

P28

%fe

wer

term

inat

ions

use,

itfe

llbe

low

the

and

$3,9

75le

ssin

orga

niza

tion

aver

age

med

ical

cost

s

Ant

icpa

ted

futu

teof

fset

of$6

,000

,000

29


Five studies were also found in the literature that examined variousdimensions of client satisfaction (Table 5). Three of these evaluationswere conducted by external providers with four examining organiza-tions based in the United States, and one study originating in Quebec,Canada. Data was collected exclusively through questionnaires thatranged in size from two to eight pages with a time frame of point in timeto two years. Sample size ranged from a random sample of 146 employ-ees who had used the program (Pearson & Maier, 1995) to 2,217 man-agers and supervisors (Johnson & Tomsic, 1999) with response ratesfrom 26.8% to 70.5% but with a mean of only 45.8%.

All of the evaluations indicated high levels of client satisfaction suchas 93% at the Chico site of the California State University (Park, 1992) torating the overall service received as 3.92 on a 4-point scale by clients ofthe Group Health Cooperative in Puget Sounc, Washington (Pearson &Maier, 1995). What were interesting were managers’ perceptions ofEAPs. In Oher’s (1993) study, while 50% of manager’s stated that theyhad referred employees to the EAP, a majority that did so reported thatthey did not observe significant improvements in performance post-EAPuse. These respondents also reported being least satisfied with the amountof support the EAP provided in assisting them documenting performanceissues. Similarly, Johnson and Tomsic (1999) reported that only 43% ofrespondents in their survey of 1,419 managers and supervisors stated thatemployees’ performance had improved after using the EAP while Park(1992) reported that 73% of 694 employees who had used the CaliforniaState University at Chico EAP reported improved performance. Whileclient satisfaction surveys are often maligned and seen as of minimalvalue these studies indicate that they have usefulness both as internaltools for organizations and also as evaluative instruments to highlight ar-eas for ongoing attention, review and development.

PROCESS EVALUATIONS

Process evaluations are the most complex forms of evaluation pro-viding information on how EAP inputs are translated into outcomes andalso assisting in gaining an understanding of why the outcomes thatwere observed arose. Nine EAP process evaluations were found in theliterature, four relating to external EAPs, one of which was a consor-tium, while the other five explored internal EAPs. It was interesting toobserve that a minority of these, three, examined American-based pro-grams while four were reviews of Canadian organizations; one was



TA

BLE

5.C

lient

Sat

isfa

ctio

n

Aut

hor

Hai

nes,

Pet

it&

Lefr

anco

isJo

hnso

n&

Tom

sic

Ohe

rP

ark

Pea

rson

&M

aier

Yea

r19

9919

9919

9319

9219

95W

orkp

lace

Mul

tiple

Site

sU

nive

rsity

Cor

pora

tion

Cal

iforn

iaS

tate

Uni

vers

ityG

roup

Hea

lthC

oop

Que

bec,

Can

ada

mid

wes

t–U

SA

Nor

thw

est,

US

AC

hico

,Cal

iforn

iaP

uget

Sou

nd,W

ash.

Wor

kfor

ceS

ize

7,00

012

,000

400,

000

EA

PD

eliv

ery

Ext

erna

lIn

tern

alE

xter

nal

Inte

rnal

HM

O

Met

hod

8-pa

gequ

estio

nnai

re2

page

sad

min

iste

red

19-it

emsu

rvey

Que

stio

nnai

redi

strib

uted

Ran

dom

sam

ple

ofad

min

iste

red

upon

tom

anag

ers

and

incl

udin

gop

en-

and

clos

ed-

toen

tire

orga

niza

tion

EA

Pcl

ient

sus

ing

com

plet

ion

ofsu

perv

isor

sen

ded

ques

tions

a4-

poin

tco

unse

lling

retu

rned

deve

lope

dto

asse

sssa

tisfa

ctio

nsc

ale

toa

third

part

ym

anag

ers’

impr

essi

ons

Stu

dyT

ime

Fra

me

2ye

ars

1ye

arP

oint

intim

eP

oint

intim

e19

mon

ths

Sam

ple

Siz

e25

02,

217

205

1,98

214

6R

espo

nse

Rat

e82

(32.

9%)

1,41

9(6

4.0%

)55

(26.

8%)

694

(35.

0%)

103

(70.

5%)

Var

iabl

esE

xam

ined

Cou

nsel

ling

serv

ices

Kno

wle

dge

ofE

AP

EA

Pus

e,ch

arac

teris

itcs

EA

Paw

aren

ess,

use,

Sat

isfa

ctio

nC

ouns

ellin

gen

viro

nmen

tU

seof

EA

Pef

fect

iven

ess,

satis

fact

ion

satis

fact

ion,

chan

ges

Impr

oved

func

tioni

ngIn

form

atio

nex

chan

gein

perf

orm

ance

Out

com

eH

igh

leve

lsof

clie

nt70

%kn

ewab

outE

AP

50%

ofm

anag

ers

used

the

83%

awar

enes

sof

EA

PO

vera

llra

ting

3.92

satis

fact

ion

repo

rted

29%

cons

ulte

dE

AP

EA

Pan

ddi

dso

mor

eth

an34

%of

facu

ltyun

awar

eC

onfid

entia

lity

3.69

27%

refe

rred

empl

oyee

once

and

wer

em

ore

seni

orLi

sten

ing

skill

s3.

67S

elf-

refe

rred

clie

nts

had

than

thos

ew

hodi

dno

t82

%sa

wE

AP

asa

Ade

quac

yof

time

low

estl

evel

sof

satis

fact

ion

88%

satis

fied

with

reso

urce

they

coul

dus

ew

ithco

unse

llor

3.56

time

take

nto

resp

ond

HR

man

ager

sdi

dno

tsee

Frie

ndlin

ess

ofH

ighe

rle

velo

fpro

blem

70%

satis

fied

with

EA

Psi

gnifi

cant

impa

ctof

EA

P93

%of

clie

nts

satis

fied

coun

sello

r3.

46re

solu

tion,

grea

ter

leve

l85

%w

ould

refe

ran

othe

rus

eof

empl

oyee

perf

orm

-w

ithth

eE

AP

Frie

ndlin

ess

ofof

satis

fact

ion

empl

oyee

ance

89%

satis

fied

with

EA

Pte

leph

one

serv

ice

hour

s3.

72S

mal

ler

the

orga

niza

tion

43%

indi

cate

dem

ploy

eeE

AP

leas

thel

pful

in91

%w

ould

use

EA

Pag

ain

Exp

ertis

eof

grea

ter

leve

lofc

lient

perf

orm

ance

impr

oved

prov

idin

gco

nsul

tatio

nan

din

coun

sello

r3.

38sa

tisfa

ctio

nre

port

edaf

ter

refe

rral

assi

stin

gdo

cum

ent

73%

repo

rted

impr

oved

perf

orm

ance

perf

orm

ance

afte

rE

AP

use

31


Scottish and one Australian. Average workforce size was considerablysmaller than found with both cost-benefit reports and client satisfactionstudies.

The process evaluations had a broad range of purposes (Table 6) in-cluding examining program use, implementation, effectiveness, pre-vention, satisfaction, cost and impact on the quality of life. Accordinglythey employed a broader range of methodologies:

• employee, client, manager and counsellor point in time surveys• key informant interviews• focus groups• document reviews• pre-post treatment questionnaires• quasi-experimental time series design• pre-test, post-test experimental design

Results obtained again generally demonstrated that the EAPs wereoperating as expected and providing benefits to the workplace and thusto both employees and employers. However, process evaluations arealso conducted to provide recommendations for future program devel-opment and most of the studies not only discussed outcomes but alsomethods to enhance existing services or procedures.

CONCLUSION

Is the glass half full or is it half empty? Thirty-nine studies in tenyears, less than four per year, a third of which were primarily descrip-tive in nature and several others which would not withstand the scrutinyof an undergraduate research methodology course. The 1990s did notprovide an extensive EAP evaluation legacy. Yet, in a field with greatpromise and much talk but at best negligible support both academicallyand organizationally for research, 39 examinations of program design isstill an accomplishment worth noting and reviewing. The practitionersand academic evaluators who went to the effort to assess need, describeprogram development and examine what programs were actually doingand then publish these findings have enhanced the baseline of knowl-edge in the EAP field, a practice area that is generally acknowledged asbeing grossly understudied.

The evaluators demonstrated creativity and ingenuity through therange of methodologies and both qualitative and quantitative data col-



TA

BLE

6.P

roce

ssE

valu

atio

ns

Aut

hor

Bla

ze-T

empl

e&

Hon

igB

urke

Gos

s&

Mea

rns

Kar

untz

oset

al.

Mac

dona

ldet

al.

Yea

r19

9719

9419

9719

9819

97W

orkp

lace

Hos

pita

lLau

ndry

and

Line

nP

ublic

Acc

ount

ing

Firm

Reg

iona

lSch

oolB

oard

Roc

kfor

dE

AP

serv

ice

Tra

nspo

rtat

ion

Com

pany

Per

th,A

ustr

alia

Can

ada

Sco

tland

Roc

kfor

d,Ill

inoi

sO

ntar

io,C

anad

aW

orkf

orce

Siz

e40

02,

150

12,5

007,

120

1,64

0E

AP

Del

iver

yE

xter

nal

Ext

erna

lIn

tern

alE

xter

nal

Inte

rnal

Stu

dyT

ime

Fra

me

2.5

year

sO

nepo

inti

ntim

e2

year

s8

mon

ths

4ye

ars

Stu

dy’s

Pur

pose

Rev

iew

impl

emen

tatio

n,ut

iliza

tion

and

awar

enes

sP

rogr

amus

ean

dsa

tisfa

ctio

nE

ffect

iven

ess

ofE

AP

prov

isio

nE

nhan

cese

rvic

espr

ovid

edto

clie

nts

Em

ploy

eean

dor

gani

za-

tion

pers

pect

ives

onE

AP

Met

hodo

logy

Qua

si-e

xper

imen

tal

Ano

nym

ous

Pre

-pos

tcou

nsel

ling

Lite

ratu

rere

view

Clie

ntsu

rvey

mul

tiple

time

serie

sor

gani

zatio

n-w

ide

ques

tionn

aire

s/sc

ales

(n=

101)

2pr

e-3

post

EA

Pus

esu

rvey

(n=

1608

)co

mpl

eted

bycl

ient

sR

evie

wof

stan

dard

(n=

241)

and

EA

Ppr

otoc

ols

Cou

nsel

lor

inte

rvie

ws

Em

ploy

eesu

rvey

(n=

136)

coun

sello

rsF

ocus

grou

psC

ompa

nyre

cord

sus

ing

Abs

ente

eism

data

with

EA

Pco

unse

llors

,a

case

-con

trol

desi

gnw

orks

itead

min

istr

ator

sF

orea

chE

AP

user

aQ

ualit

ativ

ein

terv

iew

san

dke

yin

form

ants

mat

ched

subj

ectf

ound

coun

sello

rsan

dcl

ient

sT

otal

Met

hods

Use

d2

15

33

Out

com

eE

AP

impl

emen

ted

with

a78

%aw

are

ofpr

ogra

m80

%cl

ient

sho

pefu

lat

Hire

dsp

ecia

lized

EA

P89

%of

EA

Pus

ers

utili

zatio

nra

teof

4.3%

in60

%w

ould

use

prog

ram

begi

nnin

gof

coun

selli

ngco

unse

llors

with

very

satis

fied

1989

and

3.6%

in19

90an

dth

isle

veli

ncre

ased

expe

rtis

ein

wom

en’s

87%

ofst

affw

ere

awar

e6%

had

used

prog

ram

afte

rth

efir

stse

ssio

nan

ddi

vers

ityis

sues

66%

indi

cate

dlif

eha

sof

the

EA

Pth

ough

only

and

repo

rted

tobe

61%

high

lysa

tisfie

dch

ange

dfo

rth

ebe

tter

46.3

%of

line

staf

fsai

dso

mew

hats

atis

fied

atco

nclu

sion

ofse

ssio

nsD

evel

oped

in-h

ouse

2%w

orse

ned

that

they

wou

ldus

eit

with

the

prog

ram

Incr

ease

inse

lf-es

teem

staf

ftra

inin

gan

dad

ded

3.6-

4.1

on5

item

sw

ithus

ing

Ros

enbe

rgsc

ale

new

asse

ssm

enti

tem

sJo

bpe

rfor

man

cech

ange

:R

ecom

men

dst

ruct

ural

4in

dica

ting

som

ewha

t21

%im

prov

eda

lot

chan

ges

tow

orkp

lace

satis

fied

Rep

orte

dpo

sitiv

eN

ewou

trea

chm

ater

ials

25%

impr

oved

slig

htly

outc

omes

inco

mm

unity

linka

ges

44%

stay

edth

esa

me

qual

itativ

ein

terv

iew

san

dtr

aini

ngpr

ogra

mfo

rw

orks

ites

adop

ted

EA

Pus

ers

have

mor

e62

%de

crea

sein

sick

days

and

abse

teei

sm6-

mon

thco

mpe

nsat

ion

clai

ms

post

coun

selli

ngth

anno

n-us

ers

33


TA

BLE

6(c

ontin

ued)

Aut

hor

Mow

ryet

al.

Sha

in&

Suu

rval

iS

tolz

fus

&B

enso

nY

amat

anie

tal.

Yea

r19

9719

9719

9419

99W

orkp

lace

Prin

ceE

dwar

dIs

land

Nip

issi

ng&

Tem

iska

min

gM

inne

sota

Min

ing

&U

nive

rsity

ofP

ittsb

urgh

(Can

ada)

Pub

licS

ervi

ceA

sses

smen

tRef

erra

lSer

vice

Man

ufac

turin

g(3

M)

Pitt

sbur

gh,P

enns

ylva

nia

Ont

ario

,Can

ada

Mid

wes

tern

,Uni

ted

Sta

tes

Wor

kfor

ceS

ize

7,55

244

59,

700

EA

PD

eliv

ery

Inte

rnal

Ext

erna

l/Con

sort

ium

Inte

rnal

Inte

rnal

Stu

dyT

ime

Fra

me

4ye

ars

Poi

ntin

time

inte

rvie

ws

30m

onth

s1/

4ye

ars

Stu

dy’s

Pur

pose

Pro

gram

anal

ysis

for

How

does

this

mod

elof

EA

PE

xam

ine

effe

ctiv

enes

sof

anS

ervi

cepr

ovis

ion,

effe

ctiv

enes

s,fu

ture

plan

ning

impa

ctqu

ality

oflif

eE

AP

prev

entio

nco

mpo

nent

clie

ntsa

tisfa

ctio

nan

dco

st

Met

hodo

logy

Sur

veys

:K

eyin

form

anti

nter

view

sP

re-t

est/p

ost-

test

Sur

veys

:(on

eye

ar)

Clie

nts

(n=

81)

9w

orks

ites

Exp

erim

enta

ldes

ign

othe

run

iver

sity

prog

ram

sE

mpl

oyee

s(n

=23

6)5

indi

vidu

als

usin

ga

ques

tionn

aire

to(n

=18

)M

anag

ers

(n=

53)

exam

ine

attit

udes

and

prog

ram

user

s(n

=54

0)M

anag

er’s

focu

sbe

havi

ours

grou

p(n

=18

)re

cord

anal

ysis

(fou

rye

ars)

11ke

yin

form

ants

Exp

tGro

upn

=44

5/40

5(n

=38

0)C

ompa

rison

with

Con

trol

Gro

upn

=21

4/23

4fo

urot

her

EA

Ps

Tot

alM

etho

dsU

sed

41

13

Out

com

e61

%of

clie

nts

repo

rted

AR

Sm

odel

inte

rven

esw

ithD

ecre

ases

inbi

nge

drin

king

,P

rogr

amus

elo

wer

than

wor

kpla

cepr

oble

ms

indi

vidu

al,f

amily

,wor

kan

dhe

avy

drin

kers

and

incr

ease

inco

mpa

rativ

eun

iver

sitie

spr

e-E

AP

use;

7%po

stth

eco

mm

unity

abst

aine

rsfr

omt1

tot2

tot3

and

the

oper

atin

gex

pens

ew

assl

ight

lygr

eate

rth

ough

54%

repl

ied

EA

Pw

asa

AR

Sno

tonl

yin

terv

entio

nG

reat

erin

crea

sein

expt

.m

ore

clie

nts

wer

ese

enw

orkp

lace

nece

ssity

buta

lso

ave

hicl

efo

rgr

oup

inpe

rcep

tion

ofco

mm

unity

deve

lopm

ent

man

agem

ents

kills

,em

pow

er-

Ave

.1ye

arco

st-s

avin

g/O

vera

llsa

tisfa

ctio

nbu

tm

ent,

know

ledg

eof

heal

them

ploy

ee$1

,530

.Net

cost

mor

etr

aini

ng,t

echn

ical

AR

Sbu

ilds

coop

erat

ion

risks

and

inem

ploy

eem

oral

esa

ving

sof

$321

,400

afte

rsu

ppor

tand

awar

enes

sbe

twee

nla

bour

and

cost

ofpr

ogra

msu

bstr

acte

dpr

ogra

mm

ing

need

edm

anag

emen

tN

och

ange

inei

ther

grou

pin

toba

cco

use

Clie

ntas

sess

men

tofE

AP

Pro

gram

need

sto

Foc

uson

prev

entio

nan

d2.

3-2.

7on

a�

3to

+3

scal

ebe

com

em

ore

educ

atio

nno

tonl

y64

%pa

rtic

ipan

tssu

ppor

tac

ross

8ite

ms

proa

ctiv

etr

eatm

ent

trai

ning

lead

ing

tode

liver

yof

prog

ram

toal

l20,

000

Sel

f-re

port

impr

ovem

ents

onA

ntith

esis

ofm

anag

edca

reem

ploy

ees

incl

udin

ga

peer

wor

kan

dpe

rson

alap

proa

chsu

ppor

tcom

pone

ntpe

rfor

man

ceac

ross

6ite

ms

34


lection procedures employed. However, while highlighting the robust-ness of EAP evaluation and the ongoing awareness of the importance ofconducting research, it also underscores the lack of uniformity that hasbeen developed in systematically examining this dynamic area. Thus,perhaps rather than asking if the glass is half full or half empty, the eval-uators should be congratulated for knowing that there is in fact a glassthat needs to be filled and we should all be encouraged to continue to doso in the 21st century.

REFERENCES

Berman, P., Sulsky, L., Pargament, K., Balzer, W. & Kausch, D. (1991). The role ofneeds assessment in the design of Employee Assistance Programs: A case study.Employee Assistance Quarterly, 6(3), 21-35.

Blaze-Temple, D. & Honig, F. (1997). Process evaluation of an Australian EAP. Em-ployee Assistance Quarterly, 12(4), 15-35.

Blaze-Temple, D. & Howatt, P. (1997). Cost benefit of an Australian EAP. EmployeeAssistance Quarterly, 12(3), 1-24.

Bruhnsen, K. (1994). Michigan study shows EAP clients use less sick leave, stay lon-ger. EAPA Exchange, 24(8), 11; 27.

Burke, R. (1994). Utilization of Employees’ Assistance Program in a public account-ing firm: Some preliminary data. Psychological Reports, 75, 264-266.

Carlson, R. & Cyper Bartlett, J. (1994). A religious community MAP: Modifying EAPservices for new markets. EAP Digest, 14(6), 42-45; 54.

Collins, K. (1998). Cost/benefit analysis shows EAP’s value to employer. EAPA Ex-change, 28(12), 16-20.

Conlin, P., Amaral, T. & Harlow, K. (1996). The value of EAP case management.EAPA Exchange, 26(3), 12-15.

Cooper, C., Sadri, G., Allison, T. & Reynolds, P. (1990). Stress counselling in the postoffice. Counselling Psychology Quarterly, 3(1), 3-11.

Csiernik, R. (1995). A review of research methods used to examine Employee Assis-tance Program delivery options. Evaluation and Program Planning, 18(1), 25-36.

Csiernik, R. (1997). The relationship between program developers and delivery of oc-cupational assistance. Employee Assistance Quarterly, 13(2), 31-53.

Csiernik, R. (1998). A profile of Canadian Employee Assistance Programs. EmployeeAssistance Research Supplement, 2(1), 1-8.

Csiernik, R. (2003a). Employee Assistance Program utilization: Developing a compre-hensive scorecard. Employee Assistance Quarterly, 18(3), 45-60.

Csiernik, R. (2003b). Ideas on best practices for Employee Assistance Program poli-cies. Employee Assistance Quarterly, 18(3), 15-32.

Dilts, S., Gendel, M., Lepoff, R., Clark, C. & Radcliff, S. (1994). The Colorado physi-cian health program. American Journal of Addictions, 3(4), 337-345.

Donovan, R., Kurzman, P. & Rotman, C. (1993). Improving the lives of home careworkers: A partnership of social work and labor. Social Work, 38(5), 579-585.

Rick Csiernik 35


Editorial Board. (1993). How McDonnell Douglas cost-justified its EAP. PersonnelJournal, 72(2), 48.

Every, D. & Leong, D. (1994). Exploring EAP cost-effectiveness: Profile of a nuclearpower plant internal EAP. Employee Assistance Quarterly, 10(1), 1-12.

Finke, L., Williams, J. & Stanley, R. (1996). Nurses referred to a Peer Assistance Pro-gram for alcohol and drug problems. Archives of Psychiatric Nursing, 10(5),319-324.

Flynn, C., Sturges, M., Swarsen, R. & Kohn, G. (1993). Alcoholism and treatment inairline aviators: One company’s results. Aviation, Space, and Environmental Medi-cine, 64(4), 314-318.

Goss, S. & Mearns, D. (1997). Applied pluralism in the evaluation of employee coun-selling. British Journal of Guidance and Counselling, 25(3), 327-344.

Haines, V., Petit, A. & Lefrançois, S. (1999). Explaining client satisfaction with anEmployee Assistance Program. Employee Assistance Quarterly, 14(4), 65-78.

Jerrell, J. & Rightmyer, J. (1982). Evaluating Employee Assistance Programs: A re-view of methods, outcomes, and future directions. Evaluation and Program Plan-ning, 5(3), 255-267.

Johnson, D. & Tomsic, M. (1999). Documenting the value of an EAP to administratorsin higher education: A survey of customer satisfaction by supervisors and manag-ers. Employee Assistance Research Supplement, 3(1), 2-5.

Karuntzos, G., Dunlap, L., Zarkin, G. & French, M. (1998). Designing an EmployeeAssistance Program intervention for women and minorities: Lessons from theRockford EAP study. Employee Assistance Quarterly, 14(1), 49-67.

Kelly, P. (1995). The pastoral approach: A congregational assistance program. EAPAExchange, 25(5), 22-23.

Kurtz, J., Johnson, D. & Sugden, B. (1996). The United States Postal Service Em-ployee Assistance Program: A multifaceted approach to workplace violence pre-vention. In G. VandenBos & E. Bulatao (Eds.) Violence on the job. Washington,D.C.: American Psychological Association.

Kurtz, N., Googins, B. & Howard, W. (1984). Measuring the success of occupationalalcoholism programs. Journal of Studies on Alcohol, 45(1), 33-45.

Macdonald, S. (1986). Evaluating EAPs. Toronto: Addiction Research Foundation.Macdonald, S., Lothian, S. & Wells, S. (1997). Evaluation of an Employee Assistance

Program at a transportation company. Evaluation and Program Planning, 20(4),495-505.

Maiden, R.P., Kimble, S. & Sudtelgte, D. (1993). A consortium EAP for rural employ-ers: The Princeton option. EAP Digest, 13(3), 36-41; 47-48.

Merriam, S. (1989). Case study research in education. San Francisco: Jossey-BassPublishers.

Mowry, S., Hogan, S., Moase, O., Montgomery, D. & Harper, B. (1997). Public sectorEmployee Assistance Program: Program evaluation. Charlottetown, Prince Ed-ward Island.

Oher, J. (1993). Survey research to measure EAP customer satisfaction: A quality im-provement tool. Employee Assistance Quarterly, 8(4), 41-75.

Park, D. (1992). Client satisfaction evaluation: University Employee Assistance Pro-gram. Employee Assistance Quarterly, 8(2), 15-34.



Pearson, D. & Maier, M. (1995). Assessing satisfaction and non-response bias in anHMO-sponsored Employee Assistance Program. Employee Assistance Quarterly,10(3), 21-34.

Rodriguez, J. & Borgen, W. (1998). Needs assessment: Western Canada’s program ad-ministrators’ perspectives on the role of EAPs in the workplace. Employee Assis-tance Quarterly, 14(2), 11-29.

Safyer, A., Litchfield, L. & Leahy, B. (1996). Employees with teens: The role of EAPneeds assessment. Employee Assistance Quarterly, 11(3), 47-66.

Schear, T. (1995). What is the value of your EAP? EAP Digest, 15(2), 20-23.Schefstad, A., Tiegel, S. & Jones, A. (1999). Treating a visible problem with a hidden

population: A working sports EAP in the horse racing industry. Employee Assis-tance Quarterly, 14(4), 17-32.

Shain, M. & Suurvali, H. (1997). Feeding four birds with one hand: Results of theNipissing/Temiskaming EFAP/ARS impact study. Toronto: Addiction ResearchFoundation.

Stephenson, D. & Bingaman, D. (1999). A prospective cost-benefit estimate of theU.S. Postal Service EAP. Employee Assistance Research Supplement, 3(1), 5-8.

Stoer-Scaggs, L. (1990). Faculty model and evaluation strategies in higher education:The Ohio State University EAP. Employee Assistance Quarterly, 6(1), 67-73.

Stoltzfus, J. & Benson, P. (1994). The 3M alcohol and other drug prevention program:Description and evaluation. The Journal of Primary Prevention, 15(2), 147-159.

Tesch, R. (1990). Qualitative research. New York: Falmer Press.U.S. Department of Health and Human Services. (1986). Standards and criteria for the

development and evaluation of a comprehensive Employee Assistance Program.Rockville, Maryland.

White, R., McDuff, D., Schwartz, R., Tiegel, S. & Judge, C. (1996). New develop-ments in Employee Assistance Programs. Psychiatric Services, 47(4), 387-391.

Yamatani, H., Santangelo, L., Maue, C. & Heath, M. (1999). A comparative analysisand evaluation of a university Employee Assistance Program. Employee AssistanceQuarterly, 15(1), 107-118.

Yandrick, R. (1992a). At Orange County (FL) public schools, a success formula: EAPservices + MIS capabilities = program accountability. EAPA Exchange, 22(7),24-25.

Yandrick, R. (1992b). Virginia Power’s EAP responds to management’s “need toknow.” EAPA Exchange, 22(7), 26.

Yin, R. (1989). Case study research: Design and methods. London: Sage.

Rick Csiernik 37


A Review of EAP Evaluation in the 1990s

Documents