Picking the NYT Picks: Editorial Criteria and Automation in the Curation of Online News Comments Nicholas Diakopoulos University of Maryland, College Park.

Picking the NYT Picks:Editorial Criteria and Automation

in the Curation of Online News Comments

Nicholas DiakopoulosUniversity of Maryland, College Park – College of Journalism

@ndiakopoulos | nickdiakopoulos.com | [email protected]

“NYT Picks is the most popular comment queue. We spend a lot of time tweaking that and

getting that right.”

What are criteria for selection?How can we augment moderator capability to consider more

comments?

Criteria from Literature

Negative / ExclusionPersonal attacks, profanity, abusive behavior

Positive / Inclusion Internal CoherenceThoughtfulnessBrevity / Length

RelevanceFairness / Diversity

Novelty

Argument QualityCriticalityEmotionalityEntertainment ValueReadabilityPersonal Experience

Crowdsourcing

Argument QualityCriticalityEmotionalityEntertainment ValueReadabilityPersonal Experience

Internal CoherenceThoughtfulnessBrevity / Length

RelevanceFairness / Diversity

Novelty

RQ1: Do “NYT Picks” comments reflect positive editorial criteria identified in literature?

Automation

Argument QualityCriticalityEmotionalityEntertainment ValueReadabilityPersonal Experience

Internal CoherenceThoughtfulness

Brevity / LengthRelevance

Fairness / DiversityNovelty

RQ2: Can algorithmic approaches to assessing criteria be developed?

Automated scores point towards scalable opportunities for moderation and UX…

But automation also raises questions about over-generalization across contexts, and algorithmic transparency

Questions?

ContactNick DiakopoulosUniversity of Maryland, College of Journalism

Twitter: @ndiakopoulosEmail: [email protected]: http://www.nickdiakopoulos.com

More InfoN. Diakopoulos. The Editor’s Eye: Curation and Comment Relevance on the New York Times. Proc. CSCW. March, 2015.