“Emotion in Music” organizers endeavor at Crowdsourcing task: A Multimodal Approach to Drop Detection in Electronic Dance Music Anna Aljanaki 2* , Mohammad Soleymani 1* , Frans Wiering 2 , Remco C. Veltkamp 2 1 University of Geneva, Switzerland 2 Utrecht University, Netherlands * Equal technical contributions 1
14
Embed
MediaEval 2014: A Multimodal Approach to Drop Detection in Electronic Dance Music
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
“Emotion in Music” organizers endeavor at Crowdsourcing task:A Multimodal Approach to Drop Detection in Electronic
Dance Music
Anna Aljanaki2*, Mohammad Soleymani1*, Frans Wiering2, Remco C. Veltkamp2
1University of Geneva, Switzerland2Utrecht University, Netherlands
* Equal technical contributions
2
Problem definition
• Given an electronic music excerpt, its timed comments, and labels from MTurk automatically identify whether the excerpt fully or partially contains a drop.
3
Material
• 15 second excerpts with timed comments including the term “drop”
• MPEG Layer 3 files• Metadata including the comments• Labels from the crowd• 164 excerpts with full agreement (105: full
drop; 4: partial drop; 54: no drop)• 70 excerpts with no agreement
4
Solutions
• Labels from crowdsourcing 1. Majority vote (MV)2. Dawid-Skene (DS)
• Labels from crowdsourcing + comments3. Naïve Bayesian classifier
• Labels from crowdsourcing + content4. Logistic regression
5
Using labels (wisdom of crowd)
Aashish Sheshadri and Matthew Lease. SQUARE: A Benchmark for Research on Computing Crowd Consensus. In Proceedings of the 1st AAAI Conference on Human Computation (HCOMP), 2013
6
Solution 1: Majority vote
• 3 labels each• Calculate the majority• If there is no agreement then the estimated
label is 2 (partial drop)
7
Solution 2: Dawid-Skene• Dawid and Skene proposed a method to combine a
number of uncertain decisions (clinician-patient) (1979)• The method is to calculate the confusion matrices for
every labeler using Expectation-Maximization to get estimates of these values (probabilities); initialized by majority vote.
• We then look at the probability of true response given a label from a given worker for all the three workers and pick the highest one.