User and Group Behavior in Computer Support for Collaborative Reflection in Practice: An Explorative Data Analysis - Talk at COOP Conference 2014

User and Group Behavior in Computer Support for Collaborative Reflection in Practice: An Explorative Data Analysis Michael PrillaInformation and Technology Management, Ruhr University of Bochum

User and Group Behavior in Computer Support for Collaborative Reflection in Practice: An Explorative Data Analysis

Behavior

Ideas

Feelings

Returning to experience

Attending to feelings

Re-evaluating experience

New perspectives

Change in behavior

Readiness for application

Commitment to action

Experience(s)Reflective

process Outcomes

Articulating, sharing(similar) experience(s)

Articulating, sharinginsights / outcomesShared perspectives,shared understanding

Coordinating and embedding group processes of reflection into work

Critical / counterfactual thinking

Inference, abstraction

Boud (1985)

Collaborative Reflection: Sharing, Communication


The TalkReflection App for Collaborative Reflection


Sharing and communicating in contextReferring to experiences

Experience Report

Comment


Exploring Collaborative Reflection in Practice4 Studies: Hospital, Care Home, Public Administration

Neurological Hospital (DE)“How to act professionally in demanding situations?”

Dementia Care Home (UK)“How to answer this question?”

Interns in Public Administration (UK)“How to deal with challenges?”

Parking departments (UK)“How to learn good practice from each other?”

How do people use tools supporting collaborative reflection at work?


Measuring collaborative reflection Interest

Interns Parking Care home Hospital

Participants (all/active) 23/11 12/8 9/5 6/6

Days 32 32 63 49

Experience reports read 284 421 144 153

Experience reports read by active user 25.8 52.6 28.8 18.9

Experience reports read per day 12 14.3 2.4 3.1

Experiences commented on 21 (81%) 23 (45%) 11 (73%) 18 (86%)

Experience reports read per comment 8.1 7.8 8 4.7

Length of threads 2.05 2.24 1.5 1.37


Measuring collaborative reflection Feedback



Days 32 32 63 49








Measuring collaborative reflectionQuality



Days 32 32 63 49








Measuring collaborative reflectionInterest, Feedback and Quality

Content analysis: Cases 1 and 2 outperform cases 3 and 4 in output: solutions, learning, change (Prilla and Renner 2014)



Days 32 32 63 49







Differences in the performance of groups usingcollaborative reflection support

What influences activity and quality?


Roles in Collaborative ReflectionExample: Hospital

U4.1 U4.2 U4.3 U4.4 U4.5 U4.60

2

4

6

8

10

12

14

16

76

4

21

00

7

10

5

15

Documented expe-riences

Comments

U4.1 U4.2 U4.3 U4.4 U4.5 U4.60

5

10

15

20

25

30

35

40

45

50

7

44

14

21

33

12

Shared experiences r...

Commenter:Mainly comments,

little writing

Documenter:Mainly reports, little reading / commenting

Reader: Interest in other

reports, little writing / commenting

(Typical) Reflection

participant:All activities


Comparing the groupsInterns and Hospital

U4.1 U4.2 U4.3 U4.4 U4.5 U4.602468

101214161820

76

42

100

7

10

5

15

Hospital

U4.1 U4.2 U4.3 U4.4 U4.5 U4.60

10

20

30

40

50

60

70

7

44

14

21

33

12

U1.1 U1.2 U1.3 U1.4 U1.5 U1.6 U1.7 U1.8 U1.9 U1.10 U1.110

2

4

6

8

10

12

14

16

18

20

43 3

1 1 1 1 1 1 1 1

11

98

3

1 1 1 1

InternsDocumented experiencesComments

U1.1 U1.2 U1.3 U1.4 U1.5 U1.6 U1.7 U1.8 U1.9 U1.10 U1.110

10

20

30

40

50

60

7064

2429

6

12

18 20 21

27 2833

Shared experiences readBroad, active

reflection groupSeparated roles,

low activity

Case 1 Case 4

Reports read per day 12 3.1

Avg. length of threads 2.05 1.37

Collaborative reflection needs a critical mass of activity

Groups with separated roles may

• create too little overall activity to succeed

• complement / add on each others’ activities

@ Walter Reich / Pixelio


The role of dominant usersReflection groups focused on one user

U2.1 U2.2 U2.3 U2.4 U2.5 U2.6 U2.7 U2.8 U2.90

5

10

15

20

25

30

35

40 37

2 1 2 1 0 1 1 0

32

6 5 4 3 2 1 0 0

Parking

Documented experiences Comments

U2.1 U2.2 U2.3 U2.4 U2.5 U2.6 U2.7 U2.8 U2.9-30

20

70

120

170

220 213

28 2538

2747

19 11 13

Shared experiences read

U4.1 U4.2 U4.3 U4.4 U4.5 U4.60

5

10

15

20

25

30

35

40

7 64

2 1 00

7

1 0

5

15

Hospital

U4.1 U4.2 U4.3 U4.4 U4.5 U4.6-30

20

70

120

170

220

7

44

14 2133

12

User 2.1: Driver of reflection, motivating participation, blocking other authors

User 4.6: Providing (only) answers, hierarchical position,

blocking further reflection

Case 2 Case 4

Experience reports read 421 153

Avg. length on threads 2.24 1.37


▪ Drawing conclusion 1 from Comparing stats / qualitative data 1 (e.g., lead user good/bad or critical mass and takeup)

For collaborative reflection support a dominant user may• facilitate usage• support group activity• block participation• spoil reflection


(One) Implication: Activation and prompting

▪ Foster active participation in reflection▪ Reflection mostly not implied by work▪ Create interest, e.g. make documenters aware of relevant

content▪ Foster discussion, e.g. ask readers for comments▪ Motivate sharing experiences, e.g. prompt commenters

for reports

▪ Current activity: Content analysis for identification of key contributions in reflection▪ Specific support of activities

▪ Content analyses


Thanks for your attention. Questions?

Michael Prilla [email protected] www.imtm-iaw.rub.de, www.mirror-project.eu

michael.prillaimtmmirrorip

User and Group Behavior in Computer Support for Collaborative Reflection in Practice: An Explorative Data Analysis - Talk at COOP Conference 2014

Technology

experience reports

group behavior

group processes of reflection

computer support

active user

typical reflection participant

experience new perspectives

good practice