Visualizing the Semantic Structure in Classical Music Works Winnie Wing-Yi Chan Supervised by Dr. Huamin Qu MPhil Thesis Defense Examination Department of Computer Science and Engineering The Hong Kong University of Science and Technology January 13, 2009
63
Embed
Visualizing the Semantic Structure in Classical Music …wallacem/winchan/research/thesis-slides-winnie.pdf · Visualizing the Semantic Structure in Classical Music ... begins with
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Visualizing the Semantic Structure in Classical Music Works
Winnie Wing-Yi Chan
Supervised by Dr. Huamin Qu
MPhil Thesis Defense Examination
Department of Computer Science and Engineering
The Hong Kong University of Science and Technology
January 13, 2009
Outline
Introduction
Related Work
The Musical Structural Data
Data Preprocessing
Design Principles
Visualization Prototypes
User Interactions
Case Studies
Evaluation
Conclusion
2
Outline
Introduction
Related Work
The Musical Structural Data
Data Preprocessing
Design Principles
Visualization Prototypes
User Interactions
Case Studies
Evaluation
Conclusion
3
o Background
o Motivations
o Contributions
Background
Classical music is generally less friendly and approachable
Realizing structure is difficult for unskilled ears
Learning with musical scores and essays are demanding 4
The first theme begins presented in a minor key by the violins. Then part of thefirst theme is repeated by the violins. The transition begins in the violins; itcontains a short motive and many scalewise passages. The second theme ispresented by the violins and woodwinds. The second theme is repeated by thewoodwinds and violins. The music moves on to some transitional material. Amotive from the first theme is played by the clarinet and bassoon. The codettabegins with rapidly moving notes played by the violins. The codetta concludeswith a solid chord. The development section begins with two short chords andnotes played by the woodwinds. A portion of the first theme is played by theviolins three times in sequence, each time one note lower than before. A portionof the first theme is played by the lower strings as the upper strings play acontrasting line of rapidly moving notes. The upper and lower strings tradewhat they play. The upper and lower strings again trade what they play. Theviolins continue playing portions of the first theme. The motive from the firsttheme alternates quietly several times between the violins and the flute.Fragments of the first theme are exchanged more energetically between theupper and lower strings. A short link leads to the recapitulation. The first themereturns played by the violins in the tonic key. The first theme continues. Part ofthe first theme is repeated by the violins. The transition begins…
(book covers courtesy to amazon)
Motivations
Visualizing the semantic structure of music
Impress with its sophistication and beauty
Reveal underlying musical structure
Assist music learning and teaching
Previous approaches
Rarely on semantic structure for perceptual understanding
Mainly on sonic features or low-level physical properties
5music visualizers examples
Contributions
We pioneer a visualization solution to convey the semantic structure in classical music
The inverted, shortened motif in the lower strings contrasts the music at macro-level
52
Introduction
Related Work
The Musical Structural Data
Data Preprocessing
Design Principles
Visualization Prototypes
User Interactions
Case Studies
Evaluation
Conclusion
Outline
53
o User study
o User survey
User Study
Novice users gain insights promptly without prior training
A pictorial guidance for music learning and appreciation
54
Accuracy Time (min)
Prototypes Mean SD Mean SD
Layer Braid 86% 12% 4.17 1.68
Theme Fabric (Bundled) 95% 9% 5.13 0.73
Theme Fabric (Collapsed) 89% 15% 3.67 1.83
Details 10 college students 12 questions 5-min introduction
User Survey
Braid and fabric are excellent metaphor of music
Layer braid is useful as the layer role concept is abstract
Bundled theme fabric is preferred by advanced users
Collapsed style is concise and aesthetic
Prototypes cooperate with or complement one another
55
10 Experts 7 music students 2 music educators 1 musician
Expert Feedback Highlights
Educators
• Reading essays diminishes learning interest• Layer information is important
Students
• Layer feature helps differentiate instruments• Comparing compositions become easier
Musician
• Audience find classical music more approachable• Listeners grow from appreciation to learning
56
Visualizing music without clear form?
… design highly generalized.
Improvements
Include more attributes, e.g. dynamics, tempo, tone color
Add descriptive texts
Introduction
Related Work
The Musical Structural Data
Data Preprocessing
Design Principles
Visualization Prototypes
User Interactions
Case Studies
Evaluation
Conclusion
Outline
57
o Demo
o Summary
o Conclusion
o Future Work
Demo
58
Summary
Innovative visualization solution for semantic structure in music
Macro-level layer relationships
Micro-level theme variations
Macro-micro interactions between layers and themes
Design principles
Color scheme for layers
Glyph grammar founded on typography
Visualization prototypes derived from textile art
Layer braid
Theme fabric (plain, bundled and collapsed style)
Representative case studies
Encouraging user evaluation 59
Conclusion
Apply information visualization to other discipline
Strong human factors
Artistic sensation
Introduce art and design analogies to address perceptual issues
Typography
Textile art metaphor
Generalize the techniques for applications with semantic context
News visualization
Story-telling visualization
60
Future Work
Visualize more compositions to fully realize its potential
Implement a simple user-interface for data input
Seek the possibility of using the visualization as teaching tool
61
Publications
Wing-Yi Chan, Huamin Qu, Wai-Ho Mak. “Visualizing the Semantic Structure in Classical Music Works”. Accepted to IEEE Transactions on Visualization and Computer Graphics (TVCG) with minor revision.
Huamin Qu, Wing-Yi Chan, Anbang Xu, Kai-Lun Chung, Kai-Hon Lau, Ping Guo. “Visual Analysis of the Air Pollution Problem in Hong Kong”. IEEE Transactions on Visualization and Computer Graphics (TVCG), vol.13, no. 6, pp.1408-1415, 2007.
62
Q & A
63… Thank you!
After silence, that which comes nearest to expressing the inexpressible is music.