ppt

DistributedComputing

Group

Visually and Acoustically Exploring the High-Dimensional Space of Music

Lukas BossardMichael KuhnRoger Wattenhofer

SocialCom 2009Vancouver, Canada

2 Michael Kuhn, ETH Zurich @ SocialCom 2009

• Storage media– Vinyl records– Compact cassettes– Compact discs

• An Album is stored on a single physical storage medium– Sequence of songs given by album– Album is typically listened to as a whole

History

organization by album


Music today

• Huge offer, easily available – filesharing, iTunes, amazon, etc.

• Large collections– The entire collection is stored on

a single electronic storage medium

– Organization by albums (and other lists) is no longer appropriate

organize by similarity


Organization by Similarity

• Our Goals– Mobile application (portable player)– Play songs the user likes– Overview of a collection

• Problems on mobile devices– Limited input– Limited output– Limited processing power– Limited memory

• Solution– Use song coordinates provided by www.musicexplorer.org


Which songs are similar?

• Goussevskaia et al., WI 2008:– Each song is positioned in a Euclidean „Map of Music“– Similar songs are close to each other in this Euclidean space


The Map of Music

• Based on usage data– „behaviour of the crowd“ – Gathered from social music platform (last.fm)– NO audio-analysis!

• Underlying similarity measure– Item-to-item collaborative filtering (Amazon)

[Linden et al., IEEE Internet Computing]

– „users who listen to song A also listen to song B“

• Coordinates available through webservice– www.musicexplorer.org


Hey Jude

Imagine

My Prerogative

I want it that way

Praise you

Galvanize

rock

pop

electronic

Using the Map

• Similar songs are close to each other

• Quickly find nearest neighbors

• Span (and play) volumes

• Create smooth playlists by interpolation

• Visualize a collection

• Low memory footprint– Well suited for mobile domain

convenient basis to build music software


That‘s easy – is it?

10 dimensional!


Contributions

Visual and acoustic guide to the

high-dimensional music galaxy

Proof-of-concept application for Android devices („Google-phone“)


Visual ExplorationVisual Exploration


The Reference: SensMe (Sony Ericsson)

slow

fast

happysad

Create playlist by selecting

areas

Based on audio-analysis


Requirements

Global Overview

Local Overview

Orientation

Our problem: 10 dimensions!


Lens Metaphor

Few details in the border rings

Detailed view in the center


Lens: Recursive Clustering

High resolution in the center

Few details in the border regions


Cake Metaphor

Used to represent song clusters


The Visual Exploration Interface

• Browsing– Touch cluster to bring it to the

center

• Playlist Generation– Select a number of seed songs– Playlist will consist of songs

around these seeds– Similar to SensME (but songs are

selected in a different interface)

Touch to make this area the new

center


Evaluation (1)

• User Experiment– 9 participants– Collection (1400 songs)– 5 minutes to create playlist of 20 songs (for both systems)

• Evaluation: Participants had to...– ...rate each individual song in the playlists– ...fill in a questionaire

vs.


Evaluation (2)

• Average song rating (scale: 0..10):– 5.5 (SensMe)– 6.3 (this paper)

• Questionaire (scale: 1..5):

SensMe This paper

Playlist (overall) 2.4 3.3

Diversity (3 is best) 2.4 3.4

Usability 4.7 3.7

Underlying space 2.4 4.0

Use again? 44% 67%

Trade-off:

Accurracy of high-dimensional space versus simplicity of interface


Acoustic Exploration


Idea

Shuffle

(play songs in random order)

Can we do better? Yes!Idea: Learn on the fly which songs the user likes!

Skip = bad songListen = good song


Realization

Basic algorithm: Voronoi TesselationFirst song was skipped

Supposed to be the user‘s region of

interest


Improvements

• Weighting– Account for strong/weak feedback

• Aging– Allows to adapt to changing mood

• Centering– Border regions are risky => go to center

• Escaping– Sometimes play random song to avoid getting stuck somewhere

Rating bar (left = skip, right =

good)


References

• Random shuffling (e.g. iPod-Shuffle)

• Pampalk et al. (ISMIR, 2005)– Designed for (Euclidean) audio feature spaces

dg

db

If there are songs with dg < db:

select such song with smallest dg

Else:

select song with largest ratio dg/db


Evaluation

• 9 Participants• Song ratings are used as input and for evaluation

Diversity clearly better than Pampalk

Ratings clearly better than

random


Conclusion

www.musicexplorer.org


Questions?

ppt

Documents

user likes

mobile devices

create playlist

play songs

similar songs

song

high

music