Music Research Concept Paper incl. Colour Organ Akiko Abe Graphic Design and New Media.

Music ResearchConcept Paperincl. Colour Organ

Akiko AbeGraphic Design

and New Media

Research Targets

• How minimal/conceptual music composers make music. After looking

at some basic parts of music theory, I want to understand the way

they do. And hopefully, want to try for myself.

• Also how they are different comparing with the traditional music and

composers’ context. How they broke the rules if there have been

any.

• Also listen to romantic music for comparison and investigate why it

is romantic, as I also often work in romantic motives.

• In parallel with the investigation of them, watch again the films they

contribute their music to. Make notes again. It must be different to

the previous impression and understanding. If time allows, try games

too.

Things to investigate

• What does it mean that music has mathematical aspects? Is

my mathematical knowledge helpful to the research and

making musical elements within my work? (You can’t just say it it

mathematical because we do some calculation when composing time of each notes, triads

or harmonies. Those are just arithmetic!)

• Make a colour organ (not necessarily made like an “organ”)

and enjoy playing, composing, and sharing with other people.

Try to watch the reaction.

• Feedback this to the project Blue. Current keywords are:

– Motif: Hope, Beginning, Change

– Visual: Blue, Dawn, Light, Open sky and things in the sky, Water,

Slow change

Things to investigate

Research Paths

• [R1] Reading Books

Discuss with teachers for inside topics. If the book is too difficult, ask help.

– Experimental Music - Cage and Beyond (Nyman)

To understand the context of minimal music and major composers in the field

– Sounds in Space Sounds in Time (Vella)

To learn and develop musical thinking and making in a creative way

– Music and Mathematics (Fauvel, Flood and Wilson)

To investigate the relationship between music and mathematics

– The Third Ear (Berendt)

Because I am not sure if I have the third ear, though I do have the third eye

– 大作曲家たちの履歴書 (CVs of great composers) （三枝 Saegusa ） [Japanese]

To look through some major composers’ profiles for background knowledge

– Point and Line to Plane (Kandinsky)

Because I adore this book

For others, also approach to poems. Reading voice is my favourite sound

source.

How to proceed investigation

• [R1] Case Study

– Music 　とりあえず聴く。聴く、聴く、聴く。そして聴く !!! 耳の限界に挑戦。

o Experimental/Conceptual/Minimal music <--> In parallel with filmso Nyman, Cage, Eno, Takemitsu etc.

o Romantic music <--> in parallel with films

o Beethoven, Wagner, Tchaikovsky, R. Strauss, Debussy, Preisner etc.

o Satie, looking at Klee (Satie is cited in Jarman’s “Blue” too)

o Ballet music, looking at Joseph Cornell or de Gas

o Opera/Theatre music, looking at David Hockney (go theatre if possible)

o Holst, reading Greek myths

o Poems, reading aloud

o Hymns, gospells in a church. Read a sutra. Crystal ball music

while meditating

o Collect environmental sounds: water, wind, something burning,

breaths..

How to proceed investigation

• [R1] Case Study (continued)

– Films

Check specially those in which music is powerful in evoking certain feelings

to the audience or giving impact to its storytelling. As this is connected to

films survey in Primary Research, I choose works from those in the

research.

Nyman flicks (Greenaway films, The Piano etc.)

Preisner’s works in Keislowski films (Preisner is influenced by Sibelius)

Ennio Morricone’s works in various films

Animation works: Fischinger, McLaren etc.

Horror movies

Watching CFs, music PVs

[R1] Making notes

This time, because the works vary and extensive, all the notes will be taken

in one format based on my work structure model. I am now formatting this.

How to proceed investigation

• [R2] Making Colour Organ

This work is supposed to be made with Flash based on the transplant of my

previous work [am] from Director’s lingo codes. Details are in later pages.

This activity will be done by Easter break. (if be late, until the end of the break)

– Design visual presentation

– Design musical interaction

– Define colour-sound-key (or other devices) mapping

– Design backend architecture

– Obtain the sound source

– Work in actual coding

[Proj.Blue] Working on actual sound/musical elements for visual worksThis is the temporary goal of this research but I am not sure if I can reach, for I extended

the

research very much. Constantly watched by tutors and if cannot handle for myself, ask

help.

Saving time and scceleration by learning efficiently is the key.

How to proceed investigation

Colour Organ Project

Visual PresentationAny better idea?

A little bored of such visual… wondering what should be best…

Searching for interesting visual…

I need some twist… currently almost no idea. Please help.

Searching for interesting visual…

Iwai’s work is a transformation of drawing into sound scapes, rather than a colour organ which is similar toa real musical instrument. It produces various simple-shaped figures which are coloured according to their relevant pitches sent from theconsole and went through the dot gridscreen.

As seen in the work, it stands on theparadigm that one (or more)

octave(s) are distributed to rainbow colours,probably uniformly.

I am personally still in wonder if colours should be shared by every pitch equally, without a bias.

Case Study - Piano - as Image Media (Toshio Iwai)

This is more like a graphicequalizer. Its sound is similar to that of a hammond organ but is ambient and beautiful,depending on the user’sinteraction.The user can slide each

barand create the overalltexture of the organ’ssound, according to theauthor, Pitaru. And we

cansee the polygon placed inthe centre of the screenchanging its shape.

Hammond Flower:http://www.pitaru.com/

Case Study - Hammond Flower (Amit Pitaru)

This is also rainbow-coloured and similar inthe concept as Iwai’s work. The weaving

startsfrom the centre and spirally goes

outwards.

The difference from the previous two

cases is that it has its trace of playing music as a woven tapestry, although it lacks

information

of each note’s duration.

It has three tones; accordion, trumpet andpiano, which are different in its colour

tonesin the screen. The switching key position is fixed but hidden. So is every colour-sound

key.

Case Study - O - [am]

アンデミルミル (Andemirumiru)A fancy knitting product by Hamanaka,based on Lily-yarn

リリアン (Lily-yarn)

The weaving visual is madeinspired by simple meiasweaving and very popular Japanese girls’ toy “Lily-yarn,”which almost all Japanese around my age have seen it at least once in their childhood.

If you play it by your visual curiosity, tapestries may be like them.

If you play it by your musical curiosity, then tapestries are..

Case Study - O - [am] if the key mapping is known…

Each sound x 4 for one octave Edel Weiss (Accordion) Chiisana Konomi (Trumpet)Originally the aria in “La Jolie Fille de Perth” by Bizet

Colour and Sound

Blue does not seem like being attached to a specific colour as stably as red - C,

green - F/Fsharp or magenta - A. Where is blue? Where should it be? Or it should be everywhere?

Nowhere?

Historical Colour-Sound Mapping

O - [am] Colour-Sound-Key Mapping

• Basically mapped like a piano

• Piano’s white keys are distributed onto two lower rows of the keyboard and the

upper first row of keys are for piano’s black keys, which makes the touch of

playing quite different to that of a real piano

• Right hand keys and left hand keys are same sets and independent to each

other

• Tone change keys for each hand change the tone of the sound based on the

toggle: – Right: Accordion -> Piano -> Trumpet -> Accordion -> … (repeat)

– Left: Trumpet -> Accordion -> Piano -> Trumpet -> … (repeat)

And it changes the colour tones as well (next page)

Tone: Trumpet

O - [am] Colour-Sound-Key Mapping

• This colour mapping is too blue-dominant. But the paradigm of what colour for how much has no rule originally.

Tone: Accordion

Tone: Piano

O - [am] Colour-Sound Mapping Alternative

I would rather do this!

Musical InteractionAny better idea?

Key Mapping Alternatives: 2-octave Piano

• I thought of this mapping when making [am], but discarded because it was not my aim to make it like a piano

• The upper two rows for the right hand, the lower two for the left

• It is a serious imitation of a piano keyboard but both hands will confuse each other because the space is very narrow

• Can expect fast-response interaction owing to its simplicity, but still the outputting sounds will have some difficulty if too much keys are pressed at one time

Key Mapping Alternatives: Guitar (or String-based)

• This mapping is based on how we play a guitar• Right hand is for up/down strokes or arpeggio with the base

shortcut and each string: 1,2,…, 6, while left hand for bunch of chords

• Need to prepare triad sounds in advance for strokes, as well as single sounds source for single string pickings, which may cause a delay in response. Especially stocks for choking should vary in combination and large in data size

• Yet might be very interesting in terms of interaction design. Similar kind of toy guitar (shaped like electric guitar) exists in Japan

Is there any other better mapping?

• Not always have to follow a piano just because it is a “keyboard”

• For example, xylophone is also interesting to imitate, realising fast-changing

triads

• Dependent on the sound source, the instrument can be a rhythm instrument

too

• The instrument could be something like a environmental sounds juke box with

video pieces

• If there is other controllable input device, it gives more interaction space

• Must remember that Flash can only have two audio tracks. In order to play

more sounds, multiple sound source must be given a priori, in either/both track

Interesting cases beyond a keyboard

Basically, development in interface using other devices costs high and needs advanced

engineering

knowledge and experience, therefore, it is often a collaboration with a manufacturing

company.

• ARS Electronica (probably similar works exist by previous nominees)

http://www.aec.at/en/index.asp

• TENORI-ON by Toshio Iwai and YAMAHA

http://www.global.yamaha.com/design/tenori-on/

• Maywa Denki (see ART -> TSUKUBA series)

http://www.maywadenki.com/

Maybe many of Music or New Media Department in various universities are doing similar projects.

But I don’t know who is doing what, especially in England, outside Japan.

High-IQ projects in science

• Hyperinstruments project led by Tod Machover, at MIT Media Lab

http://www.media.mit.edu/hyperins/

• Sony CSL (seems that they are doing musical stuff in France, but this lab. is more famous

for development in intuitive user interfaces.)

http://www.csl.sony.co.jp/

bye bye

Appendix

Colour-Sound Mapping: Sound Tapestry (v3)Sound Tapestry is one of my past

works, which shares the weaving

algorithm partially and the sound

set with O - [am].

It was a school project using Director, which we newly learned, and I made it with the inspiration given by early 80s video games.

Tone: Piano

Tone: Trumpet

Tone: Accordion

Sound Tapestry: Visual Samples

In Sound Tapestry, the player can choose instrumentsfor each hand separately before playing, and colourweaving is done from the most left upper corner ofthe screen. The sound source is the same set as [am]’s.

[am] has its breakthru on the cross point of a creativemotivation towards a toy for celebrating children andthe reflection that this sound set suits better for healthy visuals rather than the video-game atmosphere.But this time, I really want to renew the sound set.

Colour me any colour of you!

Please give me any suggestion about this research project and if you need, please use this

keyboard map.