Peer Feedback in Academic Writing using Google Docs2012-08-30
• Björn Hedin• KTH Royal Institute of Technology• Media Technology• Stockholm, Sweden• [email protected]
Annotation and Social Annotation
• Annotations: - “marks on (or attached to) readingmatter” including
underlining, highlighting and comments.• Different from Notes
- “marks that are not co-located with the text to which they refer”.
• Plenty of research indicate that notes and annotations are very important for learning.
• Social annotation tools add a collaboration aspect to annotations, which allows users to - share,- interact and - collaborate on annotations attached to documents.
Social Annotation Tools
• Demo- Diigo
- Google Docs/Drive
- Crocodoc
What We Have Done
• Started with peer feedback in academic writing in 2004- Either comments in separate threads or problematic
commenting in word documents• B.Sc. Thesis in 2010 investigated using SA tools instead,
promising results• 2011 – Tests in 2 groups, good results• 2012 – Google Docs much improved, full scale test• Investigated
- What are the students’ (and teachers’) attitudes to using Google Docs for Peer Feedback?
- What are the student’s attitudes to the use of screen vs. paper when reading documents?
- Which effects can recent and upcoming technical developments have on the above two questions?
Material
• B.Sc. Degree project course • Group supervision
- Meet 6 times and give written and oral feedback to each other on progress of report.
- Written in pairs. 74 students produced 37 final reports (exjobb).
- About 8-10 students per groups meaning 4-5 reports.- Hand in report about a week before seminar and use SA
tool (google docs) for Peer Feedback Marking.- Discuss on seminars, often using projector.
• Evaluated using questionnarie (60% response rate), focus groups and interviews with teachers.
• Demo
Results: Using Google Docs for Writing
• 11% of students prefer word processor over Google Docs when writing individual reports
• 89% prefer using Google Docs when writing group- or pair reports
• Main problem seen with Google Docs was possibilities to format exactly as you want. Also better dictionaries.
• Most students used google docs for writing and in the final version imported into MS Word for final layout.
Reading on Screen vs Paper
“If given a pdf-file to read before a seminar, how often do you read it on screen compared to print-
ing it on paper.”
0%
5%
10%
15%
20%
25%
30%
35%
40%
100%80%60%240%20%0%
Why Screen and Why Paper?
• Why screen?- Simpler to read it on a screen than to print it- Environmental aspects- Economical aspects- The ability to search in the texts- Easier to organize files than paper
• Why Paper?- Most mentioned a better reading experience, especially
for longer texts.- 8 of 26 mentioned better possibilities to annotate the text
But…
"How easy is it to comment using Google Docs?"
0%
10%
20%
30%
40%
50%
60%
Very EasyQuite EasyModerately difficult/easyQuite DifficultVery Difficult
Usefulness of feedbackUsefulness of getting written peer feedback, written supervisor feedback and oral feedback (scale 1-5, 5 is best)
Series10
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Written Peer Feedback (σ = 0.86)Written Supervisor Feedback (σ = 0.87)Oral Feedback (σ = 1.10
Usefulness of feedback (2)
• 4 student thought the written group feedback was more useful than the written supervisor feedback,
• 21 students thought the written supervisor feedback was better than the written group feedback,
• 13 students thought they were equally useful.
Comments in Document or in Separate Thread?
"“Is it best to comment on reports directly the document (as in Google Docs) or in a separate threaded fo-
rum?"
0%
20%
40%
60%
80%
Clearly Google DocsGoogle DocsUndecidedSeparate ThreadCleary Separate Thread
Problems?
• Not really many problems found, but maybe…- Pdf files doesn’t work well with Google Docs- Perhaps more focus on details
Benefits
• Peer Feedback Marking in general a very good peer learning activity (see research by Nancy Falchikov)- Student get insights in other theses and problems- Useful to give and receive feedback
• Comments in context• Increased interaction among the students in giving
feedback.• Simple matters (spelling and so on) easily addressed • Seminar time can be spent with well-prepared
students• Students really read and comment texts
- Difficult to get away with not reading the texts• So simple that even teachers want to use it
Future and Conclusions
• Increased use of tablets like iPad- Even easier to read and annotate than on computers
• Constantly better cloud-based word processing with annotation features
• SA tools can be used for much more!• Students and teachers were all very enthusiastic, so
it’s worth giving a try!