Collaborative Group Work: How Students Feel About the Process Robert Gervey Chia-Chiang Wang Mary O'Connor Drout Department of Rehabilitation Psychology & Special Education University of Wisconsin-Madison
Collaborative Group Work: How Students
Feel About the ProcessRobert Gervey
Chia-Chiang Wang Mary O'Connor Drout
Department of Rehabilitation Psychology & Special Education
University of Wisconsin-Madison
Study Setting RP&SE 500
Foundations of Rehabilitation Counseling An entry-level course in the Master’s program for
rehabilitation counselors. It teaches: Legislative history of profession Trends in societal attitudes and behaviors toward
individuals with disabilities Ethical issues of the profession Vocational rehabilitation systems and practices Roles and functions of rehabilitation counselor
Structure of the Course: Pre-Collaborative Group Work
Class has been structured for past several years as a: Hybrid in-class/web-based course Mix of guest speakers, didactic presentations, small group dis
cussions and five in-class debates Five debate topics
Is history an inevitable story of progress and improvement? Has the Americans with Disabilities Act been successful? Should a person who makes bad life choices be eligible for lim
ited federal-state vocational rehabilitation funding? Should the feeding tube have been removed from Terri Schiav
o? Should sheltered workshops be closed?
Rationale for Introducing Collaborative Group Work Exercises To encourage students to provide constructive criticism t
o one another so as to enhance Each participant’s level of participation in debates Each participant’s quality of participation in debates The overall team performance of the debates
To improve self-reflection and self-appraisal about group collaboration skills
To provide students with effective strategies in working with a diverse group under a high stress (time limited), high demand (grade involved) situation that approximates case planning meetings and/or multi-agency, multi-disciplinary case treatment conferences
Sample 27 students
5 master students 22 undergraduates
Gender Male-5 Female-22
12 report prior experience participating in classroom debates
Method: Assignment to Collaborative Work Groups Developed 4 teams so as to create small
groups of 6-7 students to collaborate in 5 class debates Randomly assigned students to collaborative work
groups Group membership remained constant throughout
semester
Method: Structure of Debates Debates were administered in two separate classroo
ms led by the Instructor or the TA Individual teams rotated so that each team had the
opportunity to debate each other team and also be evaluated equally by the TA and Instructor
Each team member had 5 minutes to present their initial argument and 2 minutes for rebuttal
Instructor/TA and 1 or 2 members of each team served as judges for the debate
Collaboration: Defined Week prior to each class debate
On-line threaded group discussion was mandated during which time students were to expected to plan, critique and refine their arguments for the upcoming in-class debate. Students needed to: Post early and often Comment on others postings Edit/modify their arguments based on comments fro
m others Submit written argument
document in dropbox prior to debate
Collaborative Group Work Evaluations Self and Peer Assessments
Students completed on-line (Learn at UW) assessment of collaborative group members’ behavior during week preceding class debate (5 separate assessments-one after each of the 5 debates)
Class evaluation Students completed in-class paper-pencil self report of
learning experience of just completed week of collaborative group work preparation and debate (5 separate evaluations- one after each of the 5 debates)
Course evaluation Students completed pre-post in-class course evaluation
Collaborative Group Work Monitoring: Fidelity
Instructor/TA Monitoring and Prompts Instructor and TA comments made within the
threaded discussions to prompt, probe and direct participation and guide argument formation and to provide constructive feedback about specific types of collaborative group work activity evidenced by group members
Collaborative Behavior: Feedback Report Card includes:
Self- and Peer-Assessment of Collaborative Behavior During Week Preceding Class Debate
Number of Postings Made Number of Posting Read Instructor grade for in-class debate argument Instructor grade for written argument submitted to Dropbox Peer written feedback about Collaborative Behavior across
3 Weeks of Collaboration In-Class, Small Group, Face-to-Face Verbal Feedba
ck from and to Collaborative Group Members (no Instructor or TA present)
Outcome Evaluations: Instructor Generated Verbal Performance
Instructor/TA ratings of individual in-class debate arguments (0-2 scale) (10-15 minute per student)
Written Performance Instructor/TA ratings of individual debate argument submitted to L
earn at UW Dropbox prior to debate (0-2 scale) (1-2 pages in length)
Participation Number/Percentage of Postings within Group Threaded Discussi
on Number/Percentage of Postings Read Timeliness of Postings (Latency of Response)
Results
Enjoyed collaborative debate activity?
I enjoyed the collaborative debate activity
0.0%20.0%40.0%60.0%80.0%
100.0%
Pre Debate 1 Debate 2 Debate 3 Debate 4 Post
Agreement
Increased understanding of materials?
Working in a group increased understanding of material
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Pre Debate 1 Debate 2 Debate 3 Debate 4 Post
Agreement
Helped understand the multiple perspectives on
the issue?Working in a group helped me to understand the multiple
perspectives on the issue
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Pre Debate 1 Debate 2 Debate 3 Debate 4 Post
Agreement
Like the idea of rate self and teammates in terms of collaboration process?
I like the idea of rating myself and my teammates in termsof our collaboration process
0.0%20.0%40.0%60.0%80.0%
100.0%
Debate 1 Debate 2 Debate 3 Debate 4 Post
Agreement
Feedback from teammates enhanced my writing and the strength of my debate?
The feedback from my group members enhanced mywriting and the strength of my debate
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Debate 1 Debate 2 Debate 3 Debate 4
Agreement
On-line collaborative process more efficient and productive than face-face?
I found on-line collaborative group process more efficientand productive than having to meet my team face-face
0.0%20.0%40.0%60.0%80.0%
100.0%
Debate 1 Debate 2 Debate 3 Debate 4 Post
Agreement
Level of comfort
Level of comfort with the teaching methods used in thiscourse
0.0%
20.0%
40.0%60.0%
80.0%
100.0%
Debate 1 Debate 2 Debate 3 Debate 4 Post
Level of comfort
Post: 62% in 2008 vs. 85% in 2007
Level of satisfaction
level of satisfaction with the teaching methods used in thiscourse
0.0%20.0%40.0%60.0%80.0%
100.0%
Debate 1 Debate 2 Debate 3 Debate 4 Post
level of satisfaction
Post: 54% in 2008 vs. 90% in 2007
Examples of Qualitative Assessment: Collaborative Work Group Member Feedback to Alice I appreciate your hard work and effort in the group projects. I
believe the group would benefit from an increased confidence in your work.
Alice is generally very timely with her discussion posts and is very good about commenting on others’ write-ups.
I feel she has shown to be very punctual and precise in her arguments. She brings a relaxed feel to the group which I appreciate. I feel she’s on the higher end in terms of postings and feedback.
Examples of Qualitative Assessment: Alice continued I thought you did a good job helping to organiz
e our group and you gave good feedback. Way to be on top of things!
You are always great about reading everyone’s write ups and giving constructive feedback. Very encouraging and helpful and willing to put in the extra work. Perhaps could post a little bit earlier.
Qualitative Assessments Related to Leadership: Examples of Statements Made to Different Individuals I think you’re a really great leader! You do a really good job of
posting early and providing lots of really helpful feedback for other group members.
I think your leadership is great – I would suggest using this leadership to give more feedback on discussion boards.
You take leadership where necessary. I like that you aren’t overwhelming and you get the job done.
______ did an excellent job starting off our group when no one knew exactly what was expected.
Qualitative Assessment: Organization The only suggestion I can think of might be to work on
organizing your arguments a little more: You usually find really good points and good data for backing up what you’re saying, but sometimes I have a little trouble following your train of thought.
I would like to see more organization in what you say in the
actual debate. It always seems you have a lot of great ideas, but they are not broken down into simple arguments to follow
Qualitative Assessment: Strength of Argument I feel like you really understand the debate materials and
topics and you always come up with really solid arguments. I have also noticed a few times where your arguments have
kind of seemed to summarize the points other people included in their arguments, and I feel like this is somewhat unfair to some of the other members who worked hard on their own, original arguments.
I appreciate that your debates are strong and that you do your research.
Qualitative Assessment-Timeliness
You’re really good at getting your debate points posted early
You are fantastic at very timely feedback on-line and that is very helpful when it comes to tightening up our points.
I know things are really busy and everyone’s schedules are hectic, but sometimes I feel a little bothered by how late you post your debate points, particularly if you’re asking for feedback.
Although I understand you have a very busy schedule it was difficult when you couldn’t’ make the debate on time.
Qualitative Assessment-Feedback
You seem to provide a lot of helpful feedback to other group members.
Good feedback but I think she could be a little more assertive when she has an idea
You provided some great constructive feedback
I think maybe you could provide our team members with a little more constructive feedback rather than just the blanket statement of “good job”.
Qualitative Assessment-Feedback
You’re really responsive to other people’s feedback to your arguments.
I would have liked to see more discussion from you online, and I think you, along with the rest of us, could have benefited from incorporating some feedback into your arguments.
She was helpful to others and not afraid to ask for help herself, which made me feel better about my own confusion with the material.
She did not hesitate to revised her debates based on our feedback
Other Student Comments Let someone else have the first word for once. I
admire that you get your work done quickly, but others are slower at reading and response.
I think your participation, responses, and leadership were all on the right level exactly! I’d love it if you could encourage and support some of the other group members to do the same!
I think it would be great to see your increased confidence and initiative in these projects.
Although you have great input, maybe holding off and letting others post first might be beneficial to the group.
Other Student Comments You seem like a really hard worker and I like the way you
really get involved in the debates. I think she always has a positive attitude and willing to take
on sections or arguments (when in the debate) that are challenging.
I am pleased with the rotation of duties. I see great potential in your thoughts! You do a really great job and I think you’re a really strong
team member.
Student Comments—Written vs. Verbal Feedback “Thanks for allowing us to have time to hash
this all out face to face. I will feel much more comfortable presenting my constructive criticism in person. This will also provide an opportunity to collaborate as a team to stratagize (sic) ways in which we can support one another in achieving these suggestions.”
Students’ Assessment about Assessment “I don’t really have anything to say about
anyone specific. I appreciate those who are able to lead our group and begin the discussions.”
“Overall, I am very pleased with my group and their effort. I know that this is not necessarily what Dr. Gervey was looking for; however, if there is nothing to say, why make up something?”
Self- vs. Peer- Assessment
Pattern for rating Self vs. Peers Below: 6 (23%) Even: 11 (42%) Above: 2 (8%) No Pattern: 7 (27%)
+/- 2.5 points allowed
Self-rating = Peer-ratings Debate 1: 11 (42%) Debate 2: 15 (60%) Debate 3: 13 (52%) Debate 4: 11 (44%)
+/- 2.5 points allowed
•62% of students (N=16) agreed that s/he was tougher when rating self than when rating teammates
•Subjective data
Post- Survey The requirement of having to give feedback to
my group members was helpful 42% (N=11) for agreement
Written vs. Verbal feedback from my group members enhanced my class performance Written 58% vs. Verbal Verbal 73%73%
I would like other courses that I attend to use collaborative projects as a learning method 54% (N=14) for agreement
Relationship between Process and Performance: Where is the Beef? Peer ratings of group collaboration and
instructor generated performance ratings Inconsistent findings
Significant correlations found only with Debate 2 (r = .435, p <.05) Debate 3 (r = .601, p <.01)
To Collaborate or Not: Students’ Choice For final project, students were given a
choice to collaborate Only 6 of 27 elected to work collaboratively Reasons given for working independently
52% - more flexible and time-saving 17% - the site I assessed near my home 13% - learn more by doing it solo 9% - hard to co-work with others 9% - other
Discussion
Did we actually make the student experience of the class worse by adding the collaborative group work process ?
Students’ overall satisfaction with course dropped significantly from year prior
Students voiced quite a bit of displeasure with collaborative group work process Students did not particularly appreciate quantitative report
card style feedback- reported too many numbers andstatistics
Discussion Are we over-estimating the enjoyment and utility of In
ternet in terms of replacing or augmenting in-class, face-to-face interactions? Strong student preference voiced for face-to-face verbal fee
dback Did we select the correct activity to teach and measu
re collaboration? Debates appeared to be perfectly suited
The frequency and various methods of assessment seemed excessive for both students and Instructor/TA Data overload Diminishing returns?
Implications/Future Research Questions Do we need more upfront discussion about students
’ pre-class attitudes and experiences concerning collaborative group work?
Should we expect collaborative group work exercises to yield the results that we had anticipated Enhanced class participation Enhanced quality of work Enhanced match between self and peer assessment of coll
aborative group work skills Providing students with an individual and group grad
e for their collaborative group work does not appear to resolve grading concerns
Limitations
Possible confounds Instructor/TA differences between years Cohort effect- different set of students Was it the debate or the collaborative group work
activity that was the source of displeasure Rating instruments and methods not tested
for reliability or validity
Questions?? Comments??
Thank you
Special thanks go to DOIT and ENGAGE for providing all needed consultation and technical supports.