Matpresentation2

Peer Review on a Wiki

Brandon WolfeSubmitted for completion of Master’s in Arts and Teaching

University of Arkansas, 2009

• Amplification

• Augmentation

• Group Coordination

“How can people and computers be connected so that—collectively—they act more intelligently than any individuals, groups, or computers have ever done before?”

BUT“. . .it is business as usual. Computers are enclosed in computer rooms

rather than being a central part of the learning experience…it is not necessarily lack of funds, but the lack of adequate training and lack of understanding of how computers can be used to enrich the learning experience.”

U.S. Department of Education National Education Technology Report

“Wikis have not yet made it to the classroom, either as a research topic or as a teaching method.” Evans (2006)

“Perhaps one reason that there isn't written a lot about wikis is that they are a generic tool. What makes it to the bigger journals is research on specific theories and practices. There are many things one can do with a wiki”

E. Kophler of MIT's Scheller Teacher Education Program

A Lesson!

A survey!

Results!

• Students do believe that wiki lessons help them understand why scientists rely on peer review.

• Observing my best practice guidelines for wikis improves students’ opinions of peer review still further.

• But, nothing changed student opinions more broadly.

AssumptionsThe following assumptions apply to this study:• Students will report the opinion of the unit honestly, and will not feel they are

obliged to give the lesson a high rating.• Students will participate in the lesson as it was intended. Limitations Imposed on the ResearcherLimitations on the internal validity of the study include the following:• Students will be willing to complete a survey.• Students will return a permission slip to complete the survey. This requirement

will bias the results.• Both researcher and subject are strongly biased toward affirmative results.

DelimitationsLimitations on the generalizability of the study include the following:• The study is given to a tiny set of students, leading to large statistical

uncertainties. • The study will not be repeated in different contexts, which may be particularly

important given its technological base.• Without testing the results’ reproducibility, an affirming bias cannot be eliminated.

Wikis are…• Constructive

– “We can already see a convergence between these new open media tools and constructivist learning: all three require the users to construct their own content” (Seitsinger 2006)

• Reflective– “Enabling students to integrate new ideas with prior knowledge

to make meaning and enable learning” (Miers 2004).• Scaffolding

– “With their iterative real-time redesign, are perfectly structured to provide . . . scaffolding challenges” (McGonigal 2008).

• Collaborative– “…enact knowledge building with and for others, with the focus

being on the community rather than on the individual learner” (Chao 2007).

The Standard Objection

“beacuse if i want to do a lab and submit it and everybody changes it then it wont be my project anymore and will most likly not have the same idea as i was going for in my project.”

–a student

Survey Metrics

I. Increasingly curious about scienceII. Comfortable finding peer groups that

improve their scientific understandingIII. Conversant with scientific ideasIV. Confident in their ability to produce work

that is valuable for the groupV. Likely to emphasize the importance of group

work and peer review in the life of a scientist.

I. Increasingly curious about scienceII. Comfortable finding peer groups that improve their scientific understandingIII. Conversant with scientific ideasIV. Confident in their ability to produce work that is valuable for the groupV. Likely to emphasize the importance of group work and peer review in the life of a scientist.

Figure 1: Percent Agreeing – Percent Disagreeing

Figure 4 &5:Full survey results, discriminated by period number. ‘Probability of insignificance’ is the p value of a two-way T-test, with two tails and assuming unequal variances. Only questions concerning peer review have been changed significantly.

“because i got to see other peoples and what they did on the project and how they where the same and different and what i could do next time to improve my project”

“I have the data right there in front of me and it is eisier to connect with people”

“i can see and talk about my mistakes”

It’s just easier to ‘see’ things on a wiki.

Figure 6 &7:Full survey results, discriminated by period number. ‘Probability of insignificance’ is the p value of a two-way T-test, with two tails and assuming unequal variances. This survey question was not affected by changes to the lesson.

“ i dont know. this lab wasnt the most entertaining or informative... nobody really understood the point of it. this is a very confusing website.. and still is”

“this lab did not have enough information on what to do and how to do it. I think that it was lacking information.”

“This was confusing at first!!! But then I got the hang of it we should do this on a lot of more labs.”

Despite the fact that the majority of students agreed that the wiki was easy to use, for a vocal minority the ease of use was a crippling factor.

Suggestions

• The Consumer/Producer problem– Hook activity to encourage involvement

• The ‘Where Is It’ Problem– “Scrolling [and linking] often lowers usability for kids

because they're typically more immediate than adults and tend to act on what's visible.” Neilsen (2002)

• The Thousand Questions at Once Problem– Plan in phases, assign groups and tasks

• The “It’s Cheating” Problem– Collaborative mini-tasks, rubric must be especially clear

about citation and plagiarism• The Problem Problem– Require passwords and set ground rules

Wikis and organizational change

• “There - another new blog! One more person doing what AOL and ABC - and almost everyone else - expected only AOL and ABC to be doing. Where are the time, energy, and resources coming from? The audience.” Kelly 2005

• “The idea of an organization-wide online learning system, which will facilitate all of the above, never mind obtaining the political clout or hair-raising budget to get it off the ground, is mind-blowing. But does it need to be?” Seitzinger 2006

Figure 2: Survey Results in Detail