Preparing teachers for telecollaboration: A case study Margarita Vinagre Laranjeira Autónoma University of Madrid.

Preparing teachers for telecollaboration: A case studyMargarita Vinagre LaranjeiraAutónoma University of Madrid

Introduction: Course description

• Intercultural Collaborative Exchanges in Virtual Environments

• It was delivered online as part of a Master’s Degree on ICT and Language Learning offered by the National Distance Education University (UNED)

• Second semester of the academic year 2013 (3 months)

Context and participants• 9 student-teachers • 5 were teachers of Spanish as a foreign language and, out of

those five, two were based in Colombia, two others in Cyprus and one in Spain

• 3 were teachers of English as a foreign language, all based in Spain

• 1 was a teacher of French as a foreign language, also based in Spain

• As regards gender, 6 participants were female and 3 were male. They were all native Spanish speakers, with the exception of 1 student who was Greek

• The level of experience with the use of the technology was very similar and they had little or no previous experience in telecollaboration, although they were familiar with the use of some ICT tools (blogs, wikis, skype, hangouts and google+)

Course aims

• To foster a critical stance towards the literature underlying telecollaborative learning;

• To involve participants in the process of implementing a telecollaborative exchange in order to help them move from the theory to the classroom practice;

• To integrate reflective practice (Schön, 1983; Wright, 2010). In this respect the tasks sought:

a) To foster the participants’ acquisition of the theoretical, pedagogical and technological knowledge behind telecollaboration

b) To encourage the development of the skills and awareness required by the telecollaborative teacher through hands-on experience.

Virtual Community of Practice (VCOP)

• “Groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly.” (Lave & Wenger,1991).

• These communities develop their practice through a variety of methods, including problem solving, requests for information, seeking the experiences of others, discussing developments, mapping knowledge and identifying gaps.

UNIT ACTIVITY RESOURCES PERCENTAGE OF FINAL GRADE

ALLOCATED TIME

1Introduction to CSCL & telecollaboration

(contributions and discussion will take place in the forum)

Study and discussion of relevant aspects presented in resources

Teachers’ ppt presentation, recommended readings and web pages provided by instructors

2 weeks

2Models of telecollaboration

(contributions and discussion will take place in the forum)

Comparison, analysis and critical evaluation of authentic data and samples taken from projects

Recommended readings and web pages provided by instructors

30% (this percentage reflects assessment of activities 1 & 2)

1 week

3Experiencing telecollaboration (in the wiki)

Working in groups: select, read, upload, summarize and review one article about CSCL on your wiki page. Comment and discuss articles with your group members and decide jointly on possible applications to your FL classroom

http://telecollaborationuned2013.wikispaces.com/Some articles have been uploaded onto the ‘materials’ folder but students can choose others

2 weeks

4Organizing a telecollaborative project

(in the wiki)

Decide with your group members how to organize your own exchange. You will need to include guidelines, activities and tools you would use and justify your decisions

50% (this percentage reflects assessment of activities 3 & 4)

3 weeks

5Developing tools for the assessment of telecollaboration

(in the wiki)

Design a tool that allows you to assess different aspects of telecollaboration (e.g. portfolio, learning diary, questionnaire, etc)

Recommended readings and web pages provided by instructors

20% 3 weeks

Tasks & tools

Forum (all together)

• Study and discussion of relevant CSCL aspects presented in the resources;

• Comparison, analysis and critical evaluation of authentic data and samples taken from

CSCL projects;• Answer questions for

reflection.

Wiki (two groups)

• Select, read, upload, summarize and review one article about CSCL;

• Comment and discuss articles with your group members and decide jointly on possible applications to your FL classroom;

• Decide with your group members how to organize your own exchange;

• Design an assessment tool

Contribution in the forum (example)

Discussion page in the wiki (example)

Conclusion page in the wiki (example)

Method

• The study was exploratory and attempted to identify whether those behaviours that characterize telecollaboration and that the student-teachers had read about, studied and critically analyzed during the course were reflected in their own collaboration in the online environment;

• Data was gathered from the messages on the e-learning platform forum, the six (three per group) wiki pages and their corresponding discussion pages. Due to the small sample size, answers to the end-of-course questionnaire served to provide a layer of data triangulation for the quantitative and qualitative evidence.

Procedures• Each individual message in the forum was retrieved in

HTML format. In order to facilitate analysis, posts were later converted to plain text format. In addition to content, each post contained information on the date and time of its creation and also the author’s identity.

• In the wiki, each group’s submission, comprising all versions of their 3 wiki pages in HTML format, were retrieved at the end of the course and all the comments posted on the discussion pages were also extracted.

• In addition to page content, each wiki page contained information on the date and time of its creation and also the author’s identity. In the case of the discussion pages, comments posted contained the date and author’s identity, but not the time when the message was created.

Analysis of comments in the forum

• A content analysis was carried out in order to classify the students' comments into a series of categories following a modified version of the coding scheme for content analysis developed by Judd et al (2013);

• A comment was coded into a category if part or all of it matched the description;

• Each comment was independently coded by two researchers and the results were then combined in order to ascertain number and scope of messages a) within the forum (all users) and b) from individual students.

Categories of comments from content analysis (modified from Judd et al., 2013)

Category Description

Reply A comment in response to an existing comment

Collaboration A comment that showed that the author was attempting to develop a shared understanding of some aspect of the page content.

Organization A comment that showed that the author was attempting to organize the task or workload among his/her peers

Content A comment concerned with factual content on or relevant to the target page

Editing A comment that concerned some aspect of page editing on or relevant to the target page

Individual A comment directed at an individual

Group A comment directed at the group generally

Analysis of wiki page versions

• Individual user edits were assessed by comparing successive page versions to derive the combined number of non-identical lines of text in pair comparison (e.g. version A versus B plus B versus A);

• The results were then combined to provide measures of the number, timing and scope of the edits a) within the wiki (all users), b) within both groups and c) from individual students;

• If a student created two or more successive versions of a page in a short period of time (within an hour), these were combined to form a single editing session.

Analysis of comments on wiki discussion pages

• A content analysis was carried out in order to classify the students' comments into a series of categories following a modified version of the coding scheme for content analysis developed by Judd et al (2013);

• A comment was coded into a category if part or all of it matched the description;

• Each comment was independently coded by two researchers and the results were then combined in order to ascertain number and scope of messages a) within the wiki (all users), b) within the groups, and c) from individual students.

Results: Level of contribution in the forum

The student-teachers sent a total of 55 posts to the forum, with a total of 12,010 words and an average of 585.3 words per message.

See proportional contribution of text (based on word counts) to the forum of the least and most productive student-teachers

Nature of contributions in the forum: Content analysis (1)

• [Hello María, I think it was probably that, the fact that we didn’t have an induction session with the students [in order to explain in detail the different aspects of the exchange] made things more difficult and we had to learn as we went along] (individual, reply, content)

• [Hello colleagues, I thought it was very interesting to observe how the different wikis presented by the professors show an evolution in their creation and organization] (group, reply, content)

• [… I forgot to mention that [in this process] the teachers are the facilitators not only when it comes to students’ creating knowledge but also in their development of digital literacy through the ICT tools they use….This is a great initiative don’t you think?] (group, content, collaboration)

Nature of contributions in the forum: Content analysis (2)

• There was a high level of agreement between the two coders (the inter-rater reliability coefficient was .97) with the collaboration category producing the highest level of inconsistencies;

• All comments were scored in at least one of the categories (group or individual and others as applicable) and mean scores of the coders’ ratings are shown below.

Nature of contributions in the forum: Content analysis (3)

There is little difference between the total percentage of comments that the student-teachers directed at the group in general (52.7%) and those they sent to an individual (47.2%). Almost 64% of all comments were related to content and only 5.4% to editing. As regards collaboration, 21.8% of all comments suggested collaboration between students and 9% showed task organization and planning among group members. Finally, 71% of comments were made in response to previous comments.

Timing of contributions in the forum (1)

• 42 messages were sent within the task deadline and 13 were sent past the deadline;

• The messages sent past the deadline were sent by one student who had problems accessing the wiki and asked for help in the forum.

Timing of contributions in the forum (2)

All students (100%) contributed to the forum on 1 day, and most students (80%) contributed on 2 days, with 60% contributing on 3. Over half the students contributed on 4 days and 40% of students contributed on 5 days. No student contributed to the forum on 6 or more days (in 3 weeks).

Results: Level of contribution in the wiki

In the wiki, the student-teachers carried out a total of 69 page edits and contributed a total of 700 lines (sentences) to the wiki pages, with a total word count of 17,213.

See proportional contribution of text (based on word counts) to the wiki of the least and most productive student-teachers.

Nature of contributions: Content analysis comparison wiki vs forum (1)

The majority of the student-teachers’ comments in the wiki were directed at their group generally (77.2%) rather than at an individual (22.8%). Almost 55% of comments were related to content and only 18.7% to editing. As regards collaboration, 87% of all comments suggested collaboration between students and 38.1% showed task organization among group members. Finally, 40% of comments were made in response to previous comments.

Nature of contributions in the wiki: Content analysis per group (2)

Results per group were less encouraging. Although findings in the editing and individual categories are very similar or identical in both groups, the differences in the findings relating to the rest of the categories are significantly different.

Timing of contributions in the wiki (1)

The majority of the student-teachers' contributions to the wiki were made during the last few days before the deadline, with 18 edits (26%) being made during the last week of the activity, 14 (20%) during the previous week and 7 (10%) of the edits being made after the deadline.

Timing of contributions in the wiki: Comparison between both groups

• Participants in Group 1 started working on their tasks during the first week and worked regularly throughout the entire time they were allocated for the tasks, with only one edit being made after the deadline.

• Group 2 started to work in Week 4 and had only two productive weeks, Weeks 7 and 11. The week after the deadline was also quite productive for Group 2, although page edits were carried out only by two students and not by the entire group.

Timing of contributions: Comparison wiki vs forum

Most students (100%) contributed to the wiki on 3 days and over 60% contributed to the wiki on 5 days. Some students (40%) contributed to the wiki on 6 days and 30% contributed on 7 days. A minority of students (20%) contributed on 9 days and 10% contributed on 10 days or more. No student contributed more than 13 days.

Answers to self-evaluation questionnaireThe telecollaborative teacher (TTT) Completely

disagreeDisagree Neither

agree nor disagree

Agree Completely agree

1. TTT knows the different telecollaborative models and the differences among them

2 (22.2%) 7 (77.8%)

2. TTT is familiar with asynchronous and synchronous ICT tools 9 (100%)

3. TTT knows how to use web pages in order to find partners in other countries

2 (22.2%) 3 (33.3%) 4 (44.5%)

4. TTT can explain clearly to his/her partner his/her plans and expectations for the exchange

1 (11.1%) 2 (22.2%) 6 (66.7%)

5. TTT can design the structure of a telecollaborative exchange (objectives, language use, tasks) that suit his/her students’ needs

4 (44.4%) 5 (55.6%)

6. TTT can maintain successful working relations with his/her partner during the exchange

2 (22.2%) 3 (33.3%) 4 (44.5%)

7. TTT knows how to integrate the telecollaborative exchange in the subject or course s/he is teaching

5 (55.6%) 4 (44.4%)

8. TTT can find or create tasks for the telecollaborative exchange aimed at achieving some of the learning objectives of his/her face-to-face lessons

4 (44.4%) 5 (55.6%)

9. TTT can design tasks that can foster the development of linguistic and intercultural competences

2 (22.2%) 3 (33.3%) 4 (44.5%)

10. TTT can design tasks that foster collaboration among students 1 (11.1%) 3 (33.3%) 5 (55.6%)

11. TTT can develop assessment procedures that reflect the activities and tasks carried out during the exchange adequately

6 (66.7%) 3 (33.3%)

12. TTT can provide the help (linguistic, technological, cultural, procedural, etc.) that the participants need

2 (22.2%) 7 (77.8%)

13. TTT knows how to choose the most adequate ICT tools (e.g. email, blogs, wikis, skype, etc) to achieve the exchange’s and the students’ objectives

4 (44.4%) 5(55.6%)

14. Finally, we would like to know your experience as a telecollaborative student-teacher on this course

All nine student-teachers (100%) agreed, after completing the course, that they:

a) had learned about the different models of telecollaboration,

b) had familiarized themselves with the use of asynchronous and asynchronous ICT tools,

c) could design the structure of a telecollaborative exchange (objectives, language use, tasks) to suit their students’ needs,

d) had learned how to integrate the telecollaborative exchange in the classroom and that they could find or create tasks that would facilitate attaining some of the learning objectives from their face-to-face lessons,

e) could develop assessment procedures to assess the activities and tasks carried out during the exchange,

f) were capable of providing help (linguistic, technological, cultural, procedural) to participants in telecollaborative exchanges

g) knew how to choose the most adequate ICT tools to achieve the exchange’s objectives and those of the students.

Other questions showed less agreement amongst participants, with 77.8% of participants agreeing that they knew how to use web pages to find partners and to maintain good working relations with partners during the exchange and 22.2% remaining neutral (perhaps uncertain) about this aspect. The same percentage, 77.8%, agreed that they could design tasks to foster the development of linguistic and intercultural competences, whereas 22.2% remained uncertain. Finally 88.9% of participants agreed that they could explain their plans and expectations for the exchange clearly to their partners and could design tasks that fostered collaboration amongst students, whilst 11.1% remained uncertain.

Comments from participants to questionnaire (1)

• “The level of commitment to the tasks is different among participants and one needs to be very tolerant, because sometimes this is difficult to understand and accept. On the positive side it was truly great to work with very knowledgeable partners and, in general, the group was very committed” (Student 1, 16-5-2013)

• “The experience has been very interesting because it has allowed us to experience first hand, the advantages, benefits and problems of telecollaboration. Sometimes it was very difficult organizing and deciding on work dynamics but once we overcame that difficulty, the joint working sessions and the discussions in the wiki became a great experience” (Student 2, 16-5-13)

Comments from participants to questionnaire (2)

• “A very positive experience although sometimes it is very stressful when a member of the group cannot fulfill his/her obligations by a given deadline and the group cannot move ahead because the work of each one of the group members’ is required. You need a high level of commitment in telecollaborative groups” (Student 3, 17-5-13)

• “The experience has been truly gratifying and revealing. One of the main problems I faced when thinking about implementing this type of projects was how to organize telecollaborative group work and the measures it required. Thanks to this experience I have learned to manage and collaborate in this type of projects” (Student 4, 22-5-13)

Comments from participants to questionnaire (3)

• “As a student, it has been an eye-opener to realize how it is possible to organize, elaborate, create and analyze the course topics through telecollaboration. […] I’ve realized that our traditional educational system has led us to believe that everything has to be done by the teacher, from suggestions about what to say to ideas about how to work. […] We need to become more pro-active and take the initiative” (Student 5, 25-5-13)

• “What I really missed was that constant feedback wasn’t provided by the teachers, although I know that the main objective was to achieve collaboration and receive feedback from the other students on the course. I’ve had to get used to this and now I need to keep practicing” (Student 6, 1-6-13)

Findings & discussion

• There was an expectation on our part that participants would become engaged in telecollaboration, since they were supported by the task design, the involvement of the course instructors and the functionality of the ICT tools (the wiki mostly).

• At first sight, it seemed that some of our expectations were met in the wiki, since content analysis showed that in 87% of contributions participants attempted collaboration and in 40% of cases they sought to organize or plan the joint task. Furthermore, participants made almost one comment per page edit, and most comments elicited responses or were made in response to comments, which suggested that the student-teachers’ attempts to engage the other group members were often successful.

• However, after a closer look we realized that, in a similar way to the findings from the forum, the proportion of participants’ contributions to the wiki showed some of the participants’ general tendency not to collaborate, with the least productive 50% of student-teachers contributing just over 30% of the total wiki content and the most productive 50% contributing almost 70% of the total wiki content.

• The timing and distribution of the student-teachers’ contributions to the wiki also showed that, although the student-teachers had seemingly worked regularly over the time allocated to the task, this was really down to the student-teachers in Group 1.

• Most contributions were made late in the activity and very close to the deadline, which means that participants would have had limited opportunities to interact with other members of their groups.

• Furthermore, the majority of student-teachers edited the wiki pages on 5 days, which seems a very small number to contribute over a two-month period. These findings suggest that most student-teachers were happy to collaborate from time to time in order to meet the task contribution requirements rather than develop a more equitable, consensual and comprehensive group submission which would require more regular and consistent collaboration with the other group members.

• This form of behavior is consistent with that encountered on many university courses, in which a highly individualistic approach to course work and assessment is expected.

• Clearly, most of the student-teachers on this course expected tasks to be carried out individually, as can be deduced from their initial contributions to the forum.

• Once they started working in the wiki and, after the course instructors emphasized the importance of working collaboratively, some of the participants (three student-teachers in Group 1 and two student-teacher in Group 2) did make a serious effort to contribute regularly, extensively and within the deadlines.

• Whereas most participants in Group 1 managed to achieve a certain degree of collaboration, this was not the case with the majority of participants in Group 2, who, despite reading extensively about telecollaboration and analyzing examples of good practice critically, failed to make the connection between the theory and its application to their own learning process.

• Despite these not very encouraging global findings, we did observe a positive trend towards collaboration in the wiki when we compared the data with those from the forum, in which a more individualistic approach to contributing can be observed.

Conclusions

• Despite the limitations of this study due to the small sample size, the findings suggest that the majority of the student-teachers did not apply what they had learned during the course about implementing telecollaboration to their own practice as telecollaborative student-teachers.

• One of the reasons that could account for this fact is, perhaps, that the student-teachers had different expectations about the amount of work and type of skills that telecollaboration required.

• While they had learned that they had to assume responsibility for their own work and that of others, take the initiative, contribute regularly, respect deadlines, provide feedback, discuss different aspects of the tasks jointly and reach consensus, many of them failed to do so.

• Clearly, designing activities or using technologies that are collaborative in nature does not guarantee that the student-teachers will be successful at telecollaboration.

• The findings and comments by most participants in this study also point to group management and task organization as the two aspects that were most challenging in their collaborative experience.

• As regards socialization of participants, the differences in the findings from each of the groups also suggest that, when members in a group manage to create a learning atmosphere characterized by prompt communication, regular group discussion, timely and relevant contributions, commitment to the task and consistent participation, the chances of succeeding at telecollaboration are higher.

• We have also learned that training future telecollaborative teachers through telecollaboration can help raise the participants’ awareness of what the entire process entails and, therefore, the implementation of this approach on future courses shows considerable potential for the development of the teachers’ telecollaborative skills.

• The data and analysis presented here support the idea, previously explored by authors such as Trentin (2009) and Judd et al. (2010), that the records of user activity recorded in ICT tools such as the forum and wiki employed in this project can be used to analyze and interpret the context, nature and scope of the users’ contributions.

• Finally, further research needs to be carried out on the use of automated wiki and forum data for the purpose of assessing student-teachers’ participation on future courses, either in formative or summative form.

Thanks for your attention!