Part 4 The PIC Model: Supporting Evidence or: Does it really work?

Part 4The PIC Model: Supporting Evidence

or: Does it really work?

Evaluating Prescriptive Decision Models

Descriptive models are evaluated by their empirical validity

Normative models by their theoretical adequacy

Prescriptive models are evaluated by their pragmatic value – their ability to facilitate individuals' decision-making

Evaluating Prescriptive Decision Models

The basic assumption: the right process increases the probability of choosing the best option

The evaluation of the model should examine: Does the model improve individuals' decision-making

processes? Does it lead to greater occupational satisfaction in the

future? Do individuals generalize the model and apply it to

future career decisions?

Prescreening Based on Elimination: Descriptive Validity (Gati & Tikotzki,1989)

The monitored dialogues of 384 career counselees with a computer-assisted career information system were analyzed.

Results: most users (96%) employed a non-compensatory strategy during all or at least a part of the dialogue: many options considered at a previous stage of the dialogue were not considered at the following stage, showing that individuals tend to use a prescreening strategy based on eliminating alternatives

Examine users' perceptions of MBCD

Examine changes in user’s degree of decidedness

Examine perceived benefits

Locate factors that contribute to these variables

Criteria for Testing the Benefits of Making Better Career Decisions

Method - Participants

247 males and 465 females who filled out both a pre-dialogue and a post-dialogue questionnaire

Mean age 22.8; mean years of education 12.6

4% high-school students 6% recent graduates from high school 58% recently completed their military service 9% considering an alternative to their current major 3% college graduates deliberating a job choice 8% considering a career transition 12% "other"

Method - Instruments

"Future Directions"- Israeli web site (in Hebrew)

Pre-dialogue questionnaire (prerequisite to access the system)

MBCD - Making Better Career Decisions (mean dialogue time = 40 minutes, SD=25)

Post-dialogue questionnaire

Mean Perceived Benefit (MPB) and Willingness to Recommend (WR) the Use of MBCD to a Friend (%) as a Function of the Difference in Decidedness after the Dialogue of MBCD (N=712)

Decidedness

  Increased No change Decreased

Frequency 355 (50%)

266 (37%)

91 (13%)

MPB 3.12 2.57 2.52

WR% 93.5 74.8 72.5

Measure

Frequencies of Degree of Decidedness Before and after the Dialogue with MBCD

Decidedness After the Dialogue

Decidedness Before the Dialogue

1 2 3 4 5

1- no direction 34 7 6 7 0 

2 - only a general direction

41 66 15 9 5 

3 - Client is considering a few specific alternatives

27 58 84 30 6  

4 - would like to examine additional alternatives

23 51 35 54 6  

5 - would like to collect information about a specific occupation

9 20 21 41 28  

6 - sure which occupation to choose

3 0 1 9 16 

Willingness to Recommend (WR) the Use of MBCD to a friend as a Function of the Degree of Decidedness Before and After the Dialogue with MBCD (N=712)

Decidedness Before the Dialogue with MBCD

Decidedness After MBCD 

1 2 3 4 5

1- no direction 

38 

14 17 

29  

--

2 - only a general direction 85 73 67 67 100

3 - considering a few specific alternatives

100 93 82 97 100

4 - client would like to examine additional alternatives

100 92 100 82 100

5 - would like to collect information about a specific occupation

100 

85 

90 

98 

89 

6 - Client is sure which occupation to choose

100 

-- 

100 

100 

81 

Taxonomy of Career Decision-Making Difficulties (CDDQ; Gati, Krausz, & Osipow, 1996)

Prior to Engaging in the

Process

Lack of Readiness due

to

Lack of motivatio

n

Indeci-sivene

ss

Dysfunc-tional beliefs

During the Process

Lack of Information

about

Cdm proce

ss

Self Occu-patio

ns

Ways of obtaining info.

Inconsistent Information due

to

Internal conflict

s

Externalconflic

ts

Unreliable

Info.

MBCD’s Effect on Reducing Career Decision-Making Difficulties (d, Cohen, 1992)

Scaled

Lack of Readiness

Motivation

General indecisiveness

Dysfunctional Beliefs

.31

. 13

.29

. 16

Lack of Information About

The Process

The Self

Occupational Alternatives

Additional Sources

.72

. 48

. 45

. 78

. 20

Inconsistent Information

Unreliable Information

Internal Conflicts

External Conflicts

. 11

. 18

. 01

.-13

Total CDDQ. 65

MBCD’s Effect (d, Cohen, 1992) on Reducing Career Decision-Making Difficulties

(Gati, Saka, & Krausz, 2003)

0.31

0.72

0.11

0.65

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Lack ofReadiness

Lack ofInformation

InconsistentInformation

Total CDDQ

d

Monitoring the Dialogue

Evaluating the input The 3 facets of preferences (relative importance of

aspect, optimal level, willingness to compromise) Crystallization of preferences (differentiation,

consistency, coherence) Evaluating the process

Which options were used and in what order (almost compatible, additional search, why not? what if? Compare occupations, similar occupations)

Evaluating the outcome (list of career alternatives) The number of alternatives on the list The similarity among the alternatives on the list

Perceived Suitability of the "Promising Alternatives" List (N=693)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

26+(n=37)

16-25(n=46)

11-15(n=40)

8-10(n=45)

7(n=236)

6(n=121)

5 (n=71)

3-4(n=74)

2 (n=23)

Number of Alternatives (n - of users)

too long

suitable

too short

Predictive Validity of MBCD

Design: Comparing the Occupational Choice Satisfaction (OCS) of two groups:

those whose chosen occupation was

included in MBCD’s recommended list

those whose chosen occupation was not

included in MBCD’s recommended list

Method - Participants

The original sample included 123 clients who used MBCD in 1997, as part of their counseling at the Hadassah Career-Counseling Institute

Out of the 73 that were located after six+ years, 70 agreed to participate in the follow-up: 44 women (64%) and 26 men (36%),aged 23 to 51 (mean = 28.4, SD = 5.03)

InstrumentsMBCD Questionnaire: clients were asked to report

their field of studies, their satisfaction with their occupational choice (scale of 1 – 9): “low” (1-4), “moderate” (5-7), “high” (8-9)

Procedure the located clients were interviewed by

phone, six+ years after visiting the career-counseling center

Method

84%

38%

16%

44%

18%

0%10%20%30%40%50%60%70%80%90%

100%

accepted

recommendations

did not accept

recommendations

low satisfaction

medium satisfaction

high satisfaction

ResultsFrequencies of Occupational Choice Satisfaction by Acceptance and Rejection of MBCD's Recommendations, Based on Sequential Elimination

Frequencies of Occupational Choice Satisfaction by the Search-Model Whose Recommendations Were Accepted

3 1013 10

102

23 51

0%10%

20%30%40%

50%60%70%80%

90%100%

Elimination Conjunction Compensation None

lowsatisfaction

mediumsatisfaction

highsatisfaction

Conclusions

Accepting the recommendations of the sequential-elimination-based search of MBCD produces the best outcomes (i.e., highest levels of satisfactions with the occupation)

The data does not support the effectiveness of the compensatory-based search

The data does not support any advantage of using the conjunction list over using only the sequential-elimination-search list

Alternative Explanations

Differences in the lengths of the lists

No difference was found in the OCS between clients whose list included 15 or fewer occupations and clients whose list included more than 15 occupations.

Therefore, this explanation can be ruled out.

Alternative Explanations (cont.)

Clients who accepted MBCD’s recommendations are more compliant, and therefore more inclined to report a high level of satisfaction.

However, following the compensatory-model-based recommendations did not contribute to the OCS.

Therefore, this explanation can be ruled out too.

Conclusion

Following the recommendations of the sequential-elimination-based search of MBCD produces the best outcome

Gender Differences in Directly and Indirectly Elicited Career-Related Preferences(Gadassi & Gati, 2007)

Method Participants: 226 females (74.1%) and 79

males (25.9%) who entered the Future Directions Internet site

Age: 17-30, mean=22.84 (median = 22, SD = 3.34) Years of education: mean=12.67 (median 12, SD =

1.48)

Instruments

Future Directions (http://www.kivunim.com) Making Better Career Decisions (MBCD,

http://mbcd.intocareers.org) The preference questionnaire: this

questionnaire imitated the preference elicitation in MBCD Participants were presented with 31 aspects, and were asked to rank-order them according to importance, and to report their preferences in all 31 aspects

Preliminary analysis

Lists of occupations. We used MBCD to generate three lists of occupations according to:

1. sequential-elimination 2. compensation

and, for 235 participants,

3. the list based on the conjunction between the sequential elimination and the compensatory search lists

Preliminary analysis

Determining the degree of gender-ratings of occupations was based on the judgments of 10 undergraduate students. 1 – “most (that is, over 80%) of the individuals who

work in this occupation are women” 5 – “most (that is, over 80%) of the individuals who

work in this occupation are men – over 80%" The inter-judge reliability was .96

We computed the mean gender-ratings of the lists of occupations for each participants

Means of the Femininity-Masculinity Ratings According to Type of List and Gender

3.18

2.96

3.13

2.71

2.42.52.62.72.82.933.13.23.3

ExplicitElimination

Men

Women

Gender Differences in Directly and Indirectly Elicited Preferred Occupations (Gadassi & Gati, 2007)

MBCD - Summary of Major Findings

Most users reported progress in the career decision-making process

Satisfaction was also reported among those who did not progress in the process

Users are “goal-directed” – the closer they are to making a decision, the more satisfied they are with the MBCD

Using the MBCD contributed to a decrease in career decision-making difficulties related to a lack of information

Following the MBCD’s advice doubled the probability of high occupational choice satisfaction 6 years later

PIC is compatible with people’s intuitive ways of making decisions

Summary of Workshop

Career counseling may be viewed as decision counseling, which aims at promoting making better career decisions

The PIC model facilitates the complex process of career choice by separating it into a sequence of well-defined tasks

MBCD is a unique combination of career information system, expert system, and a decision-support system based on the rationale of PIC

Summary of Workshop (cont.)

The use of the PIC model (and MBCD) contributes to: progress in the decision process, reduction in decision-making difficulties, and higher occupational satisfaction in the future

PIC and MBCD can be incorporated into career-counseling interventions

END

• sofsof

Results: Compared Means of the Femininity-Masculinity Score According to Type of List and Gender

3.182.71

3.04 3.23 3.133.22.95 2.96

11.522.533.544.55

Posit

ive

Elim

inat

ion

Compe

nsat

ion

Conj

unct

ion

fem

inin

ity-

mas

culin

ity

rati

ng

male

female