Wiki sym '12 presentation
Post on 08-May-2015
150 Views
Preview:
DESCRIPTION
Transcript
Etiquette in Wikipedia: Weening New Editors into
Productive Ones
Wikimedia FoundationRyan Faulkner, Maryana Pinchuk, Steven Walling
Making Wikipedia more welcoming to new editors with templates
Warning Templates in Wikipedia
https://meta.wikimedia.org/wiki/File:First_msg_new_users_proportional_highres.png
Why are warning templates interesting
● 4K new user accounts daily
● 1K go on to make at least one edit
● 20K new users receive at least a first warning per month
● 80% of all first messages are delivered by automated or
semi-automated tools
● Huggle was responsible for 10-40% of first messages
from 2008 to mid 2011 in any given month
How does Huggle work?
http://en.wikipedia.org/wiki/File:Huggle.png
Examples of Issue Specific Warnings
test edits - addition of an edit as a test (not content)
spamming - addition of an external link to the body of an article
unsourced content - new content added to an article without a clear source
deletion - removal of a portion of the content of an article without explanation or a clear reason
Method: The Hypothesis & Experimental Treatments
Friendlier and clearer templates generated by vandal fighting tools can increase the productivity of new editors
Method: Our Eligible Users
In order to be considered for measurement:
1. The warning received must be a first warning2. The editor must not go on to be blocked after the
warning3. The editor must be registered (not an anonymous user)
Experiments ran Nov. 8th, 2011 to Dec. 9th, 2011 inclusive
Method: Our DataFor each editor in the experiment we measure:
1. Timestamp of the warning template event2. User ID and user name3. The number of revisions in all namespaces over the editors lifetime
before and in the three day period after the template event4. The number of warnings in all namespaces over the editors lifetime
before and in the three day period after the template event5. The number of blocks in all namespaces over the editors lifetime
before and in the three day period after the template event
Method: Derived Metricsjafter(u):Edits for editor u in the three day period after the warning
jbefore(u):Edits for editor u in their lifetime before the warning
Gn:
Editor group defined by all editors making a minimum of n edits
m(u) (normalized edit difference):(jbefore(u) - jafter(u)) / jbefore(u)
This is a metric used to incorporate information about an editor's past experience
logit(template(u)) = ß0 + ß1 * m(u)The regression model used to evaluate editor productivity
E1: The editor is delivered a "shortened" message warning message.
E2: The editor is delivered a ``personalized'' message, that is, uses the active voice, explicitly acknowledges that the edit was one made in good faith, and invites discussion on the reverter's talk page.
E3: Combines both a "personalized" and "shortened" warning, and also tests issue specific warnings (This experiment only involved editors receiving warning types:test, delete, spam, unsourced)
Method: Our Experiments
Results: Logistic Regression
Experiment test sample
control sample
Editor Group
ß1 error p-value AIC
E1: short 26 44 G5 -2.151 0.9864 0.0315 88.418
E2: personalized 29 35 G5 -1.496 0.5577 0.135 85.815
E3: mixture w/ specific warnings
32 32 G5 -1.4906 0.5964 0.0124 89.088
Results: Plots
Conclusions
1. We observed a positive effect on normalized edit
difference for the experimental templates. There is clear
merit integrating this approach with existing automated
and semi-automated warning tools
2. We reported our results to the community and instituted
new warnings based on our findings.
3. Follow-up: test a wider set of treatments, try to push
editors to productivity threshold
The End
Thanks.
top related