Attitude Changes When Using Wikipedia in Higher Education

Attitude Changes When Using Wikipedia in Higher Education

Pernilla Josefsson Media Technology and Interaction Design, CSC

KTH Royal Institute of Technology Sweden

[email protected]

Olle Bälter Media Technology and Interaction Design, CSC

KTH Royal Institute of Technology Sweden

Katarina Bälter

Department of Medical Epidemiology and Biostatistics Karolinska Institutet

Sweden

Stephanie Bonn Department of Medical Epidemiology and Biostatistics

Karolinska Institutet Sweden

Abstract: In this paper we present a case study including 23 students at Williams College, MA, using social media technologies in learning activities during the winter semester 2013. The study was designed to evaluate the students’ attitudes before and after participating in collaborative wiki assignments. Results from the study showed a statistically significant positive shift in attitudes before and after using the wiki. Results indicate that the students perceived the use of wiki technology as an added value when used for learning activities in a collaborative way and designed to match the learning goals of the course. The study contributes insights into how student attitudes towards social media in higher education are closely connected to their perception of what can be interpreted as human behavior in collaborative learning activities. Keywords Wiki technology, Wikipedia, attitudes, e-learning, higher education, collaborative learning Introduction In this study, Wikimedia-related technologies in collaborative learning environments were approached to explore students’ attitudes when used in two courses given at Williams College, Massachusetts, US. The learning activities involving Wikipedia assignments were designed specifically to match the course goals (see for instance Biggs & Tang 2011). Although there are several published studies related to wiki technology in higher education, few of them report on how student attitudes change in relation to using the technology in learning activities designed to match course goals. More often, studies have explored changes in students’ perceptions in relation to online writing (Miyazoe & Anderson 2012), collaborative writing (Pifarré & Fischer 2011), and collaborative learning (Judd, Kennedy & Croppers 2010). The overall aim of this study was to explore if a change occurred in students attitudes towards the use of a wiki as it was implemented in their learning environment. To capture changes in attitudes, a pre- and a post-test assessment of attitude were done.

Wiki Technology Wikimedia is a collection of collaborative projects owned by the Wikimedia Foundation; some of their projects include Wikipedia, Wiktionary and Wikinews, of which the online encyclopedia Wikipedia is the most well known (Wikimedia foundation 2014). Characteristic for these projects are that they are collaborative, that all contributions are published with a Creative Commons Attribution 3.0 license, and that the technology can be installed and adapted outside the Wikimedia Foundation. From here on we use wiki without distinguishing between different implementations. Background E-learning Beyond the Transmission Approach The integration of e-learning technologies into the context of higher education has mostly been considered as a process to support traditional modes of learning (Laurillard 2007). These modes of learning have been described to be determined by the practice that students take part in, most often seen as teacher-centered where students are regarded as passive receivers of knowledge (Duffy & Jonassen 1992; Säljö 2005). One big difference between traditional modes of learning and e-learning is that the latter is better suited for a content-driven environment (Littlejohn et al. 2007). In order to take advantage of the content-driven environment offered, the transmission approach, which has been predominant in e-learning, must shift focus in three areas: from information to communication, from a passive to a more interactive engagement, and from individual learning to more socially situated learning (Conole 2007). Karasavvidis (2010) describes the dominant learning paradigms to be incompatible with the skills and knowledge required by the e-learning environment. This also makes wikis, which are collaborative of nature, one of the more promising e-learning tools, requiring active engagement to facilitate constructive learning. Wiki Technology in Higher Education Earlier research has shown that wikis can be used to support a wide range of educational activities, such as collaborative writing (Trentin 2009) and collective learning (Nicol, Littlejohn & Grierson 2005; Wheeler, Yeomans & Wheeler 2008). In a study, Karasavvidis (2010) found associations between better course performance and more positive attitudes towards collaboration in previous literature. Important factors that have been reported include: familiarity with the technology, careful planning for implementation and use, class size, and motivation of students to engage (Raman, Ryan & Olfman 2005). Regardless of the potential for different types of educational activities, earlier research suggest that the key to successful integration of wikis in higher education lies in how students perceive the use of them. For instance Conole (2007) argued that how students will accept this is not to be taken for granted, as the technologies they use for entertainment purposes might not necessarily imply acceptance for learning purposes. Research by Rick and Guzdia (2006), Ma and Yuen (2008), and Elgort, Smith and Toland (2008) have also shown that users do not necessarily enjoy participation in wiki activities. Similar concerns have also been raised in a study by Huang and Nakazawa (2010) who emphasized that teachers need to encourage students so that they participate in wiki activities. In contrast to commonly mentioned benefits, Karasavvidis (2010) also found that students failed to collaborate using a wiki and that they were reluctant to edit other students contributions. Karasavvidis emphasized that students compared the use of wiki with the more traditional university teaching, the latter offering them the possibility to be passive recipients of information, e.g. listening to lectures. Two identified problems were “mode of work” and “student resistance”. Mode of work was explained to be related to students’ acceptance of social networking technologies for learning purposes in higher education, compared to the use of the same technologies for entertainment purposes. Students resistance implied that they did not always want to work using a wiki, instead they often favored individual work over online collaboration.

Similar conclusions of mode of work and resistance were drawn in a qualitative study by Cole (2009), who presented possible pitfalls of integrating social networking technologies in higher education. For instance, students’ acceptances of social technologies such as wikis were perceived differently when used in an educational context compared to when used for ordinary personal use. This could explain why students showed little interest in adopting these technologies in their educational context. In a study by Ebner, Zechner and Holzinger (2006) it was shown that almost no students edited Wikipedia articles, even when the teacher encouraged this. Furthermore, while acknowledging that wiki editing was a cognitive effort that supported students learning, students seemingly would not put any effort in helping the community without getting an immediate reward. It has also been shown that most students find it unimaginable to work on a voluntary basis in order to help the entire wiki community (Holzinger & Motschnik-Pitrik 2005). Description of the Project The study presented here was conducted during the winter semester 2013 at Williams College, Massachusetts, US. It included students from two courses within different subjects. Altogether, 35 students enrolled in the courses and were invited to participate in the study. After subject-introducing lectures about the course topics, a short introductory lecture to Wikipedia and how to edit the wiki was given to the students in each course by the Wikipedia Campus Ambassador at Williams College. Ambassadors are volunteers who, among other duties, give in-class presentations about Wikipedia norms and editing basics (Wikipedia Education program/Ambassador 2014). A reason for choosing Wikimedia was the possibility to offer the students a collaborative environment to enable discussions and peer-learning within a course. Another important aspect was that we wanted to introduce the students to a tool that they can use outside the context of these specific courses, and even after they have finished their education. The introduction of wiki technology was established in consensus and accordance with the educational approach of the course responsible teachers. Therefore, the wiki assignments in the courses strongly related to their learning goals. Students in each course were given three different assignments that were to be completed in the wiki. The first was individual and the students were asked to create an account and a personal presentation page. The second and third assignments were performed in groups of 3-4 students. The groups were asked to post a solution to a problem that differed between all groups, and to write constructive comments to a solution posted by another group. In one of the two courses, a fourth and fifth assignment were completed in a similar way. The problem in the additional assignments was essentially the same, but with different data sets to work on. For the purpose of this study, three types of data were collected: 1) answers to two questionnaires, of which the first was distributed and collected before the wiki assignment commenced (pre-test), and the second upon completion of the assignment (post-test), 2) interviews with students after the courses had finished, and 3) statistics from the wiki assignments. The study was approved by the college’s ethical committee. The pre-test questionnaire was disseminated to all 35 students enrolled in the two courses. In total, 23 students responded. The post-test questionnaire was distributed to and answered by the same 23 participants. Both questionnaires were anonymous but coupled in order to identify changes in attitude for the individuals during the course. The pre-test questionnaire included questions concerning the students’ background of using social media technologies and their attitudes towards the use of wikis, both outside of Williams College and in relation to their education there. Questions included both predefined choices and open-ended response alternatives. The post-test questionnaire had three questions, identical to three of the questions from the pre-test, assessing the students’ changes in attitude. Three semi-structured student interviews were conducted after completion of the courses to elucidate the other data collected previously. Each interview lasted for 30-40 minutes and offered the students a chance to elaborate on the results of the questionnaires. The interviews were held and annotated by a researcher without connection to the student.

Data was also gathered directly from the wiki and included both the statistics of each student’s contributions and the entries from editing. Results were summarized to explore to which extent the students had contributed to the wiki during the course, as well as to investigate trends in the amount of contribution. Results and Analysis Questionnaire data were either categorized by the response type or coded as explained below. Open answers from the questionnaires were manually filtered and ascribed to three independent categories (the answers could belong to more than one category): 1) technology and design, which included answers related to technology or the design of the wiki service, such as the comment “Technology barrier for people who aren’t strong computer users”; 2) human behavior, which included answers related to the students or others’ behavior, such as the comment “De-personalized communication with people that don’t know each other very well tend to be awkward (...)”; and 3) neutral, which included answers considered as neutral to the other categories, such as the comment “I’m not sure”. To ensure a higher reliability, the filtering process was also performed by three independent researchers. First Questionnaire: Pre-Test All 23 students that responded to the first questionnaire reported that they use social media. In average, they have been using social media for 5.4 years. They spend 2.1 hours per day on social media, of which 1.4 hours have their attention directed primarily towards the applications, i.e., actively viewing, creating, or engaging in contents. When asked if their use of social media support their studies at Williams College, 52% answered “yes”, 13% answered “yes, but only when it is compulsory”, 30% answered “no”, and one student answered “not sure”. The students considered social media to support their studies mainly in four areas (extracted from a list of 13 response options): for communication with peers and teachers, for finding information, for coordinating activities, and lastly, for actual collaboration. They were also asked to rate the three social media they most often use, and the following were mentioned, in descending order: Facebook, Twitter, Wikipedia, and Glow. Facebook and Twitter were by far the most common. Wikipedia or Glow were mentioned by seven students, which may sound few considering that the course would use Wikipedia and also that Glow is the learning management system used at Williams College (Glow is an implementation of Moodle that includes a wiki feature; see http://oit.williams.edu/help/glow), but some of the students would on the other hand perhaps not consider these to be among social media technologies. Students were also asked to rate their attitudes towards the use of wikis in a learning situation along a 5-point scale from “very negative” to “very positive”, see Fig. 1 (the figure also plots the results from the second questionnaire). More than 70% of the students were “positive” or “very positive” towards using wiki in learning situations.

Figure 1: Distribution of attitudes towards the use of wikis in a learning situation. The plot shows responses

0 2 4 6 8

10 12 14 16 18

Very Negative Negative Neither positive nor negative

Positive Very positive

Pre-test Post-test

collected both before and after the wiki assignment. A complementary open-ended question about their opinions on wikis in a learning situation was also included and the responses showed an overall cautious positive view, consistent with the question about their attitudes. An example of what was interpreted to be a cautious positive view is the comment: “Interesting… never tried before. Could be a great learning tool, it seems, if executed well”. To the question about which weaknesses the students saw with wikis in learning situations, the answers could be interpreted to belong to the above mentioned independent categories “technology and design” (8 answers) and “human behavior” (15 answers), while 4 answers could be considered “neutral” (see Fig. 2 where the results from the pre- and post-tests are plotted together). In many of the answers, the weaknesses were related to concerns about trust. For “human behavior”, trust would mean for instance the correspondence between interest and activity: less interested students would contribute less than the more interested students would. For “technology and design”, trust was connected to uncertainty about the system, for instance if the information in the wiki was reliable.

Figure 2: Weaknesses of using wiki in a learning situation. Number of responses from the students are ascribed to three categories. The plot shows responses collected both before and after the wiki assignment. Answers to the question about the strengths of using a wiki in a learning situation were again ascribed to the categories “technology and design” (18 answers), “human behavior” (5 answers), and “neutral” (2 answers). The results, presented in Fig. 3, showed that strengths were mainly attributed to the technology behind the wiki. A common answer was the possibility to distribute and edit information. In the category “human behavior” the answers concerned thoughts about learning, such as “Showing ideas, helping classmates learn, learning by explaining to others”.

0

2

4

6

8

10

12

14

16

18

Technology and design Human behavior Neutral

Pre-test Post-test

Figure 3: Strengths of using wiki in a learning situation. Number of responses from the students are ascribed to three categories. The plot shows responses collected both before and after the wiki assignment. Second Questionnaire: Post-Test In the second questionnaire, the students were again asked about their attitudes towards using wikis in the course, and responded using the same predefined alternatives. The results are plotted together with the pre-test attitudes in Fig. 1. More than 90% of the students were positive. Complementing this question, the students were also asked to freely express their opinions about using wikis. Compared to the pre-test, the post-test responses reflected a more positive view with less caution. The results were consistent with the question about attitudes and included answers such as: “Pleasantly surprised by effectiveness of wiki contributed to dialogue between classmates”. In the post-test questionnaire, when students were asked about which weaknesses they could see in using wikis in a learning situation, more saw weaknesses in the “technology and design” category (16 answers), while fewer in “human behavior” (9 answers), see Fig. 2. In general, responses concerning technology and design weaknesses were more detailed compared to the answers in the pre-test. An answer related to editing the wiki, and of the kind considered to be more detailed, was: “Formatting pictures was difficult”. In the category “human behavior” most of the answers reflected thoughts about collaboration, such as: “The lack of in-person contact still feels a little strange - working on a group project while all of the participants are in different physical locations still strikes me at as a little odd (largely because I am not used to it)”. When asked about strengths regarding the wiki use in the second questionnaire, even here there were more answers in “technology and design” (19 answers) and fewer in “human behavior” (5 answers), see Fig. 3. Results in the post-test showed that the students’ perception of both weaknesses and strengths mostly concerned what could be categorized to belong to technology and design. Comparison Between the Pre- and Post-Test Responses In the pre-test questionnaire, weaknesses were mostly perceived to concern human behavior while strengths were mainly attributed to the technology behind the wiki. In the post-test questionnaire, a change in concerns had occurred and the students now attributed most of the weaknesses and strengths to the technology and design. Responses provided by students in the post-test questionnaire were also more detailed compared to the pre-test questionnaire. There was a change in attitudes among the students referring to the use of wiki between the pre- and post-test questionnaires also seen from the grading along the 5-point scale, with eight students having become more positive, and one student more negative. The remaining retained the same attitude towards the use of wikis. The difference in average “attitude value” before the implementation of wiki (3.6) and after (4.0) is statistically significant (T-test,

0  2  4  6  8  10  12  14  16  18  20  

Technology and design Human behavior Neutral

Pre-test Post-test

p<0.05). The changes in attitudes are shown in Fig. 1. Interviews The interviews were used without categorizing the responses. The three students participating in the interviews (S1-S3) confirmed the positive attitudes towards the use of wiki shown in the questionnaires. When asked more specifically about their own opinion, they expressed a cautious positive view, or in the words of S1 “quite positive”. When asked to elaborate on the use of wiki, S1 and S2 compared it to the Glow system. S1 considered the biggest difference between the systems to be the openness. One have to have an account created by the college to access Glow, while creating an account for viewing and editing the wiki was open to everyone. The openness offered in the wiki was, mainly, perceived to be beneficial. Some of the described advantages were for example how the value of the writing process increased when there was a public to write for. S3 explained that a public written assignment “makes you think twice”. Another mentioned benefit was that they had to learn what S1 called “light programming skills” in order to participate, which gave them the know-how to make future contributions to wikis. A third positive aspect, described by S2 as the best part of the assignment, was that the openness permitted feedback even from others outside of the class, which provided them with different perspectives on the same issue. Although the students mainly appreciated the openness in the wiki technology, concerns were also raised by S3 regarding that anybody could edit the wiki. Another disadvantage mentioned by the students, concerning the openness, was the absence of restricted groups, which they usually have in the Glow system. Hence, S1 perceived Wikipedia great for group work where the assignments were designed to be different for each group, as in these courses, while Glow was preferred for individual assignments or when a similar assignment was distributed to all groups. When shown results from the questionnaires, S2 and S3 agreed on the general view that they learned by reading the other groups’ wiki-pages. S2 explained further how the wiki was used to read pages and connected sources, comparing it to an index in a textbook. This would improve the research part of writing, as it became “part of the thought process”. The students also perceived that it was easier to comment and to ask questions directly in the wiki environment, as compared to sending ordinary emails. The possibility to track changes, in terms of having access to all page revisions, was even mentioned by S1 as something positive as it gave the teacher an overview of each of the students’ contributions. Data From the Wiki The data collection from the wiki was based on the contributions from all students who completed the questionnaire. The data showed that the average student edited the wiki 8.5 times and contributed with 2900 characters. However, the median was 4.0 times for editing and 2123 characters in terms of contribution, indicating that there were a few students who contributed to the majority of the edits while the main part of the students only edited the wiki four times or less, see Fig. 4.

Figure 4: Distribution of the number of edits made in the wiki by all the course participants. The most active student made 52 edits. Regarding character counts, the data collected from the wiki statistics only reveal the sum of each student’s added and subtracted characters. Thus, if a student made two edits, one adding text and one deleting text, the total character count can also be negative. Hence, the sum of characters is not a good measure of the student’s actual contribution. Furthermore, there are uncertainties concerning collaboration such as two students working together with one text while only being logged into one wiki account, or one student using more than one account. Therefore, the number of edits made may seem to be a more reliable measure of contribution than the numbers of added characters, but not without errors. In general, it is assumed that students ensure that the teachers can see their contributions. Discussion The study explored students’ attitudes towards using wiki technology in their education. The results show a statistically significant positive change in attitudes (p<0.05) before and after use of the wiki technology. In the comparison between the pre- and post-test questionnaires, the results also show that students’ perception of weaknesses and strengths had shifted. The students in the pre-test were more concerned about how other students would act (human behavior), showing trust issues, and at the same time attributing most strengths to the wiki technology. The post-test showed that the students instead perceived both weaknesses and strengths in connection to the technology. Another interesting observation is that in the post-study questionnaire, only one answer could be assigned to the “neutral” category, while the total amount of comments remained almost constant. That indicates a shift from having no specific opinion to expressing one. Even if we did not set out to analyze the student contributions to the wiki specifically, we noticed that the project compares well to other studies described in the literature: On the one hand we saw a positive attitude change and it seemed the students enjoyed the assignments, while on the other hand the amounts of contribution reflect all the tendencies concerning willingness to participate in collaborative learning and wiki activities mentioned above (Rick & Guzdia 2006; Ma & Yuen 2008; Elgort, Smith & Toland 2008; Ebner, Zechner & Holzinger 2006; Holzinger & Motschnik-Pitrik 2005). Our results are also in line with and build upon earlier research such as Karasavvidis (2010), showing that modes of work and students resistance are important aspects to consider for understanding how students will perceive the wiki as a learning tool in their education. We could also confirm the findings from Raman, Ryan & Olfman (2005), as we saw a clear shift in attitude towards being more positive as the students gained more knowledge about the technology. This study is, however, limited by the relatively small number of participants, all sampled at Williams College, as

0

2

4

6

8

10

12

14

16

1<5 5<10 10<15 15<20 20<25 >26

well as the short time frame for data collection. As the generalizability of our results may be limited due to this, larger studies with other populations may determine if our results are valid also in other settings. Future studies should also, if possible, be longitudinal and extend over more than just one school semester. Although this study has its limitations, it serves as a basis for further exploration concerning attitude changes in student populations. The study contributes with insights into how student attitudes are closely connected to their perception of what can be interpreted as human behavior, in this case most often expressed as concerns about how they and other students use and edit wikis in collaborative learning activities. Acknowledgements The authors would like to express gratitude to the students participating in this study and the Wikipedia Campus Ambassador at Williams College, Nathan Thompson. References Biasutti, M., & El-Deghaidy, H. (2012). Using Wiki in teacher education: Impact on knowledge management processes and student satisfaction. Computers & Education, 59(3), 861-872. Biggs, J., & Tang, C. (2011). Teaching for quality learning at university. Berkshire, UK: Open University Press & McGraw-Hill Education. Cole, M. (2009). Using Wiki technology to support student engagement: Lessons from the trenches. Computers & Education, 52(1), 141-146. Conole, G. (2007). Describing learning activities: Tools and resources to guide practice. In H. Beetham & R. Sharpe (Eds.), Rethinking pedagogy for a digital age: Designing and delivering e-learning. London: Routledge. Duffy, T. M., & Jonassen, D. H. (Eds.). (1992). Constructivism and the technology of instruction: A conversation. Hillsdale, NJ: Lawrence Erlbaum Associates. Ebner, M., Zechner, J., & Holzinger, A. (2006). Why is Wikipedia so successful? Experiences in establishing the principles in Higher Education. In Proceedings of I-KNOW, Vol. 6, 527-535. Elgort, I., Smith, A. G., & Toland, J. (2008). Is wiki an effective platform for group course work. Australasian Journal of Educational Technology, 24(2), 195-210. Hadjerrouit, S. (2014). Wiki as a collaborative writing tool in teacher education: Evaluation and suggestions for effective use. Computers in Human Behavior, 32, 301-312. Holzinger. A., & Motschnik-Pitrik. R. (2005). Considering the Human in Multimedia: Learner-Centered Design (LCD) & Person-Centered e-Learning (PCeL). In Mittermeir, R. T. (Ed.), Innovative Concepts for Teaching Informatics. Vienna: Carl Ueberreuter. Huang, W. H. D., & Nakazawa, K. (2010). An empirical analysis on how learners interact in wiki in a graduate level online course. Interactive learning environments, 18(3), 233-244. Judd, T., Kennedy, G., & Croppers, S. (2010). Using wikis for collaborative learning: Assessing collaboration through contribution. Australasian Journal of Educational Technology, 26(3), 341-354. Karasavvidis, I. (2010). Wiki uses in higher education: Exploring barriers to successful implementation. Interactive Learning Environments, 18(3), 219-231. Laurillard, D. (2007). Preface. In H. Beetham & R. Sharpe (Eds.), Rethinking pedagogy for a digital age: Designing and delivering e-learning. London: Routledge. Littlejohn, A., Cook, J., Campbell, L., Sclater, N., Currier, S., & Davis, H. (2007). Managing educational resources. In G. Conole & M. Oliver (Eds.), Contemporary perspectives in e-learning research: Themes, methods and impact on practice. London:

Routledge. Ma, W. W. K., & Yuen, A. H. K. (2008). A qualitative analysis on collaborative learning experience of student journalists using wiki. In Fong, J., Kwan, R., & Wang, F. L., (Eds.), Hybrid Learning and Education, Lecture Notes in Computer Science Vol. 5169, 103-114. Berlin Heidelberg: Springer. Miyazoe, T., & Anderson, T. (2012). Discuss, reflect, and collaborate: A qualitative analysis of forum, blog, and wiki use in an EFL blended learning course. Procedia-Social and Behavioral Sciences, 34, 146-152. Nicol, D., Littlejohn, A., & Grierson, H. (2005). The importance of structuring information and resources within shared workspaces during collaborative design learning. Open Learning: The Journal of Open, Distance and e-Learning, 20(1), 31-49. Pifarré, M., & Fisher, R. (2011). Breaking up the writing process: how wikis can support understanding the composition and revision strategies of young writers. Language and Education, 25(5), 451-466. Raitman, R., Augar, N., & Zhou, W. (2005). Employing wikis for online collaboration in the e-learning environment: Case study. In Third International Conference on Information Technology and Applications (ICITA'05), Vol 2, 142-146. Raman, M., Ryan, T., & Olfman, L. (2005). Designing Knowledge Management Systems for Teaching and Learning with Wiki Technology. Journal of Information Systems Education, 16(3), 311-320. Richardson, W. (2010). Blogs, wikis, podcasts, and other powerful web tools for classrooms. California, USA: Sage, Corwin Press. Rick, J. & Guzdial, M. (2006). Situating CoWeb: A scholarship of application. International Journal of Computer-Supported Collaborative Learning, 1(1), 89-115. Säljö, R. (2005). Lärande i praktiken: ett sociokulturellt perspektiv. Stockholm, Sweden: Norstedts Akademiska Förlag. Trentin, G. (2009). Using a wiki to evaluate individual contribution to a collaborative learning project. Journal of Computer Assisted Learning, 25(1), 43-55. Wheeler, S., Yeomans, P., & Wheeler, D. (2008). The good, the bad and the wiki: Evaluating student-­‐‑generated content for collaborative learning. British Journal of Educational Technology, 39(6), 987-995. Wikimedia Foundation. (2014). Wikipedia Education program/Ambassador. Retrieved 2014-04-17 from: http://en.wikipedia.org/w/index.php?title=Wikipedia:Education_program/Ambassadors Wikimedia Foundation. (2014). Wikimedia Foundation. Retrieved 2014-04-17 from: http://wikimediafoundation.org/w/index.php?title=Our_projects