Aggregating Labels in Crowdsourcing Data

CERN Openlab Summer Students

Lightning Talks Sessions

Maria Priisalu

› 19/08/2015

2

Crowdsourcing

› Many workers

› Easy tasks

› Multiple answers

› Little or no prior knowledge

› Aggregate answers

19/08/2015Maria Priisalu

Note Image from Windows Clipart

3

Advantages of Crowdsourcing

› Costs less

› Faster

› Educate the worker

19/08/2014First Name and Family Name

Note: Images from Windows Clipart

4

Research in Crowdsourcing

› Hybrid systems

› Task scheduling

› Learning algorithms

› Aggregation

19/08/2015Maria Priisalu

Note: Image from Windows Clipart

5

Our implementation

› Dawid-Skene model (EM)19/08/2015 Maria Priisalu

6

Future (and current) work

› Aggregation of other types of labels

› Sociological aggregation

› Educating the crowdworker

19/08/2014Maria Priisalu

Note: Image from Windows Clipart

Maria Priisalu 7

The Webfest

19/08/2014