Page 1
1/15www.janclaes.info
Jan ClaesSupervisors UGent : Geert Poels & Frederik Gailly
Supervisors TU/e : Paul Grefen & Irene Vanderfeesten
Investigating the process of process modeling and its relation to modeling quality
The Role of Structure Serialization
Page 2
2/15www.janclaes.info
Study 1: visualization
Research Objective 1Build knowledge about
how people create models
Overall objectiveCuriosity-driven
Build knowledge about PPM
Research Objective 2Build knowledge about
relation with quality
Research Objective 3Build knowledge about
structured modeling
Study 3: theorizationStudy 2: exploration
Research Objectives
Page 3
3/15www.janclaes.info
CREATE_ACTIVITYCREATE_START_EVENTCREATE_END_EVENTCREATE_ANDCREATE_XORCREATE_EDGE
Study 1 – Visualization
MOVE_ACTIVITYMOVE_START_EVENTMOVE_END_EVENTMOVE_ANDMOVE_XOR
DELETE_ACTIVITYDELETE_START_EVENTDELETE-END_EVENTDELETE_ANDDELETE_XORDELETE_EDGE
NAME_ACTIVITYRENAME_ACTIVITYNAME_EDGERENAME_EDGE
Page 4
4/15www.janclaes.info
CREATE_ACTIVITY CREATE_START_EVENT CREATE_END_EVENT CREATE_AND CREATE_XOR CREATE_EDGE MOVE_ACTIVITY MOVE_START_EVENT MOVE_END_EVENT MOVE_AND MOVE_XOR DELETE_ACTIVITY DELETE_START_EVENT DELETE-END_EVENT DELETE_AND DELETE_XOR DELETE_EDGE NAME_ACTIVITY RENAME_ACTIVITY NAME_EDGE RENAME_EDGE
Study 1 – VisualizationPPMChart
time
mod
el e
lem
ents
Page 5
5/15www.janclaes.info
FastmodelingSlow
modelingInitialdelayMany
pauzesFewelementsMany
elements No(separate)lay-outing
Quicklay-outingDedicated
lay-outingphase
Continuouslay-outingUnpaired
eventcreation
Pairedevent
creationNo pauzes
SerializationPaired
gatewaycreation
Delayededge
creationChunked
modeling
Study 1 – Visualization
Based on dataset of 357 unique modeling executions
Page 6
6/15www.janclaes.info
Study 1 – VisualizationEvaluation
Sample of intended users (6 academic researchers) Five extreme examples in PPMChart or Dotted Chart Observe and measure amount, quality, and timing of
insights gained through the visualization Observe and ask about perceived usefulness
Results Perceived as useful More cognitive effective than Dotted Chart
Page 7
7/15www.janclaes.info
Study 2 – Exploration
Based on dataset of 40 unique modeling executions
Fastmodeling
Slowmodeling
Initialdelay
Manypauzes
Fewelements
Manyelements
No(separate)lay-outing
Quicklay-outing
Dedicatedlay-outing
phase
Continuouslay-outing
Unpairedevent
creation
Pairedevent
creation
No pauzesSerializationPairedgatewaycreation
Delayededge
creation
Chunkedmodeling
Page 8
8/15www.janclaes.info
Study 2 – ExplorationStructuredness Movement Speed
Based on dataset of 40 unique modeling executions
Fastmodeling
Slowmodeling
Quicklay-outing
Dedicatedlay-outing
phase
Continuouslay-outing
Serialization
Chunkedmodeling
CONJECTURE 1
CONJECTURE 2
CONJECTURE 3
Page 9
9/15www.janclaes.info
Study 2 – ExplorationConjecture 1: Structured modeling
results in more understandable models
Conjecture 2: A high number of move operations results in less understandable models
Conjecture 3: Slow modeling results in less understandable models
Page 10
10/15www.janclaes.info
Study 2 – Exploration
Based on dataset of 103 unique modeling executions
T-testt=-2,231
(p=0,028)
T-testt=2,199
(p=0,030)
CONJECTURE 1structuredness
T-testt=-1,984
(p=0,049)
T-testt=0,457
(p=0,648)
T-testt=-2,183
(p=0,031)
T-testt=2,505
(p=0,014)
CONJECTURE 2movement
CONJECTURE 3speed
Page 11
11/15www.janclaes.info
Study 3 – Theorization
Based on dataset of 103 unique modeling executions
T-testt=-2,231
(p=0,028)
T-testt=2,199
(p=0,030)
CONJECTURE 1structuredness
T-testt=-1,984
(p=0,049)
T-testt=0,457
(p=0,648)
T-testt=-2,183
(p=0,031)
T-testt=2,505
(p=0,014)
CONJECTURE 2movement
CONJECTURE 3speed
WHY?
Page 12
12/15www.janclaes.info
Study 3 – Theorization
Combined
Flow-oriented Aspect-oriented
Undirected
Based on dataset of 118 unique modeling executions
Page 13
13/15www.janclaes.info
Study 3 – Theorization
A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B
learning style
degree of serialization
adopted serialization style
field-dependency need for structure
– +
course of intrinsic cognitive load for process modeling phases
course of intrinsic cognitive load for aggregation phases
course of cognitive overload
course of intrinsic cognitive load for strategy building phases
+ + +
serialization style fitstructuredness of serialization– – – –
1 2 3
Page 14
14/15www.janclaes.info
Study 3 – Theorization
Evaluation of utilityNovelty
(uses existing theories in fundamental new way)Parsimony
(11 constructs, 15 associations)Consistency
(can explain additional observations)Plausibility
(accurate and profound explanation)Credibility
(building blocks are established theories)Transferability
(problem solving in general)
Consistency based on dataset of 143 unique modeling executions
Page 15
15/15www.janclaes.info
FUTURE WORK
Knowledge gaps
Knowledge contribution Research instrument
Knowledge gap 4How to change one’s modeling strategy?
Knowledge gap 3How should one model
in a specific context?
Study 1: visualizationContributions
A – PPMChartB – 22 patternsC – 13 observations
Study 2: explorationContributionsD – 8 patternsE – 3 conjecturesF – 1 metric
Knowledge gap 1How do people
currently model?
Knowledge gap 2Which strategy is
intrinsically better?
A BC
D E F
H HG I
Study 3: theorizationContributions
G – 5 stylesH1 – 6 observationsH2 – 3 impressionsI – SPMT
I