UNIVERSITY OF SURREY Musical Illusions: Applying the Psychology of Auditory Perception to Modern Composition. Peter Bryant 5/22/2012 Abstract: For well over a century the fields of music and psychology have been largely incompatible, the focus on the arts and sciences has been in creative and scientific research and focused by the demand for original research. In this paper I attempt to address music and psychology through the overlapping tool of musical illusions. These phenomena have the unique ability to function both as a neuroscientific and psychological research tool and an auditory specimen with which to inform compositional practice. I shall identify specific areas of the brain that are responsible for the processing of auditory information and provide evidence as to how this information is processed. I shall compose two original works Collateral and ULTRA, during the course of my research, to demonstrate the applications and uses of music psychology in the field of composition and also provide examples of other compositions that have used similar techniques. I hope to also demonstrate how principles from the field of composition such as orchestration and acoustics can contribute to music psychology research for the benefit of both fields.
50
Embed
Musical Illusions and their application in Modern Composition.
My undergraduate thesis on Musical Illusions and their application in two simultaneously created compositions - ULTRA (for small ensemble) and COLLATERAL (for piano duet). Completed in May 2012 at the University of Surrey.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
UNIVERSITY OF SURREY
Musical Illusions: Applying the Psychology of Auditory Perception to
Modern Composition. Peter Bryant
5/22/2012
Abstract: For well over a century the fields of music and psychology have been largely incompatible, the focus on the arts and sciences has been in creative and scientific research and focused by the demand for original research. In this paper I attempt to address music
and psychology through the overlapping tool of musical illusions. These phenomena have the unique ability to function both as a neuroscientific and psychological research tool and an auditory specimen with which to inform compositional practice. I shall identify specific areas
of the brain that are responsible for the processing of auditory information and provide evidence as to how this information is processed. I shall compose two original works
Collateral and ULTRA, during the course of my research, to demonstrate the applications and uses of music psychology in the field of composition and also provide examples of other compositions that have used similar techniques. I hope to also demonstrate how principles
from the field of composition such as orchestration and acoustics can contribute to music psychology research for the benefit of both fields.
Musical Illusions: Applying the Psychology of Auditory Perception to
The psychology and cognition of music is a remarkable process, and arguably the
most important1, sophisticated2 and holistic3 sense that humans possess. Hearing involves
perception, cognitive processing, and a neurological response from a diverse range of brain
areas and processes4. These processes are so sophisticated and rely on such as diverse areas
that research on how we perceive and experience auditory information is only in the primitive
stages of development. The result of any perceptual experience is conscious awareness. Since
the complex and paradoxical field of consciousness is by far from exhaustively researched
there are still a great many challenges to be overcome until the ‘black box’ of consciousness
(Plomp, 2001) is capable of generating a satisfactory theory of how music functions on a
neurological basis5.
However, there is no shortage of supporting anecdotal evidence, the majority of this
originated from patients suffering from brain trauma or other ailments that, through their
symptoms and issues, begin to shed light on what the brain is actually accomplishing in order
to process sounds. By their nature as bizarre and paradoxical perceptual experiences, musical
illusions from a significant part of this evidence and by collating this, we can begin to build a
rough picture of the brain mechanisms underlying perception and begin to generate
hypothesise to test our theories. This testing of hypotheses generated throughout the last
century has already had a huge boost from recent technological advances such as FMRI and
EEG studies allowing the rigor of the scientific process to accompany the anecdotal evidence
1 See (Rees, 2010, p. 1) for a brief discussion of the survival importance of auditory information and (Changizi,
2011, p. 29) for a more in depth discussion and anecdotal evidence from a blind individual. 2 (Plomp, 2001) 3The argument that one particular side of the brain is more ‘musical’ than the other died long ago in academic l iterature. See my chapter on Brain Lateralisation for a more comprehensive review. (Levitin, 2008, p. 9) 4 Music is the most complex sense we possess in terms of the amount of brain areas that process simple tones let alone entire songs or symphonies (Ball, 2010, p. 241). 5 See (Thompson, 2009).
and increase interest from a wide range of related but disperate disciplines. (Ball, 2010, p.
241; Toga, 2000, p. 365; Jordain, 1997, p. 286).
One of the best ways to demonstrate auditory illusions is by using and explaining
certain visual illusions, as the cognitive mechanisms for both sensory experiences are very
similar. Continuing this line of development throughout this paper will help to better
comprehend auditory illusions by connecting both types of illusion together both for the
purposes of understanding and to demonstrate their similarities and differences.
A Composers Guide to the Brain
The Brain and Music
Sound enters the ear from the external world and transforms into an electrical signal
though the processes of the eardrum and cochlea. This electrical signal passes along the
auditory nerve diverging into and passing thorough many intermediate processing areas on its
way to the auditory cortex. Once there the signal spits further into many parts and is
distributed to many different areas of the cortex that adopt designated tasks in order to
process the sound as quickly and efficiently as possible. Neuroscientific research has
uncovered a large amount of evidence about which areas, processes and functions within the
brain are involved in the cognition of music with the use of brain imaging techniques such as
fMRI (functional magnetic resonance imaging) (Levitin, 2008, p. 126). However, the use of
such techniques is still proving to be a controversial, difficult, and subjective issue in modern
brain science with some psychologists such as Professor Richard Wiseman believing that the
technique obfuscates and distracts from genuine research and is purely a means to an end.
This opinion is present even among neuroscientists with Vincent Walsh believing that the
very act of an experiment affects the outcome and therefore can only provide limited results
and conclusions (Walsh, 2012). Part of the problem is that the data has to undergo
interpretation by both computers and humans in order to construct meaning therefore
involving a large element of subjectivity (Jordain, 1997, p. 284). There is also a large amount
of research conducted on animals purely for the (comparatively) relaxed ethics, the
conclusions of which contribute to the theory on human cognition (Toga, 2000, p. 366) when
the debate of whether animals are conscious or not is still debatable (Blackmore, 2005, p.
123). This is highly problematic; even when there is a consensus among researchers that the
data captured is of a reliable quality, there is still the requirement of further data (replication)
to back up the claims provided due to issues of resolution and the difficulties in drawing solid
conclusions from dynamic subject matter (Ball, 2010, p. 241; Changizi, 2010, p. 19). Despite
these difficulties, brain scans are reasonable tools for demonstrating the distribution of brain
areas on a macroscopic scale. Therefore, we now have a limited, macroscopic, and general
understanding of the neurology of the brain when we experience musical information
(Jordain, 1997, p. 286). Neuroscience does not currently seem like a promising avenue for
further investigation on the topic of musical illusions. According to Dr. Tim Hughes we are
decades away from understanding the nuances of musical processing from a neuroscientific
level6. Part of the reason is that every brain processes the same information differently
depending on genetics, experiences, or other genial circumstances. (Walsh, 2012, p. 6).
Furthermore, most of the research usually originates from abnormal patients who have
suffered neurological injuries in very specific areas of their brain and consequently lost a very
distinct musical (or other) attribute that has then, post hoc, been attributed to that area (Sacks,
2011, p. 106). These issues will develop and become exacerbated as the 21st century
progresses now that brain imaging no longer requires an autopsy. Other methods such as MRI
scanning that provide detailed images of brain structures are now available with the added
benefit that patients can be alive and conscious and therefore results can be achieved with
much less effort. (Deutsch, 1998, p. 656; Jordain, 1997, p. 286).
6 Personal correspondence. 2nd May 2012.
Music and Neuroscience
Auditory processing is not a linear process but a pluralistic and many facetted set of
systems. It is so complex that it is well beyond the scope of this paper to address the vast field
in anything approaching a comprehensive review. The simplified description outlined below
focusses on building a perspective of understanding the mechanisms that may contribute
towards the perception of musical illusions.
There are two derivatives of the auditory pathway; the ascending auditory pathway
(AAP) and the descending auditory pathway (DAP). Broadly speaking, AAP describes how
sounds from the outside world are perceived by the eardrum, digitized into electrical signals
by the cochlear, and then processed by differing parts of the brain. DAP describes the same
process and is parallel to the AAP but in reverse and functions as a filter on the perception of
sounds (specifically amplification or gain) enabling us to focus on specific aspects of sound
or blocking some sounds almost entirely7. (Rees, 2010, pp. 12 ; Meller, 2000, pp. 76, 89,
122). Auditory processing is the only sense that has this ability to focus and control the
perception of a stimulus in real time; there is no haptic, olfactory, gustatory, or visual
equivalent that has such a large degree of control over the perceptual experience (Rees, 2010,
pp. 1, 9). The auditory nerve passes modules that refine the electrical signal until it reaches
the top of the auditory pathway, the auditory cortex within the Sylvain fissure of the bran,
right in the centre of both hemispheres.
7 For a more comprehensive overview see ‘Oxford handbook of Auditory Science’ Vol. 2 by Adrian Rees.
Angular Gyrus associated with language, metaphor, and mathematics (Ramachandran, 2003)
Supramarginal Gyrus associated with language processing and perception ( (Gazzaniga, 2009)
Broca's area associated with language and speech perception (Gazzaniga, 2009).
Wernicke's area associated with language and speech perception (Gazzaniga, 2009).
Primary Auditory Cortex the main processing house for auditory information. (Beament, 2003, p. 93)
Figure 1: The Auditory Brain (main areas).
The brain spits the electrical signal into many constituent parts; the path is often
simplified broadly into two distinct but related mechanisms. The auditory nerve is the first of
these to encounter an incoming sound. This nerve then has the remarkable property of
carrying signals in either direction with 50,000 fibres per ear for incoming signals and 1800
for descending signals (Hodges, 2011, p. 103). From this point, the pathway becomes
extremely sophisticated as the auditory nerve splits and combines signals from both ears
eventually arriving at the auditory cortex where one hundred million brain cells somehow
reconstruct the electrical signal into a conscious experience of sound (Hodges, 2011, pp. 99 -
109). However, the parallel processing does not stop here as connected to the auditory core
region are many areas of the brain (the auditory association cortex) that also contribute
(Koelsch, 2012, p. 13) to the perception of musical sound including areas that at first do not
appear to have a strong connection to music such as the visual or motor cortices.
Figure 2: Further areas of the brain involved in the perception and processing of auditory stimuli. (Levitin, 2008)
The arrangement of the auditory pathway is so autonomous that there is evidence that
it continues to function when the perceiver is unconscious or suffered significant brain
trauma where all other senses have ceased to function (Sission, 1990; Liberati A. et al, 2009;
Owen, 2010; Sautoy, 2009). The layout is also extremely logical with many cognitive maps
that accurately represent the incoming signal. Such tonotopic organization has been
extremely useful in understanding the processing of pitch and has led to speculation that a
tonotopic map aids the processing of timbre (Levitin, 2008, p. 43).
The perception of frequency (and therefore the psychological effect of pitch) is the
most researched area of musical cognition8. Early research discovered pitch is actually
mapped tonotopically within the brain with a line of neurones firing at the exact frequency
that caused the eardrum to vibrate. This is a direct parallel to the function of the cochlea;
hairs move in sympathy at and around the specific pitch of incoming sounds generating an
8 See the Oxford Handbook of Music Psychology chapter 5 (Hallam, 2008, pp. 47 - 58).
electrical signal. This arrangement and firing of a specific set of neurones within the auditory
cortex explains why the range of our musical instruments only covers a quarter of what we
can actually perceive; above 5 KHz, the neurones can no longer satisfactorily fire at a
consistent rate thus leaving us with a less concrete sense of pitch. In fact, this issue goes
further than this when the maximum firing rate of any one particular neurone is a 250 - 2
KHz (Gerasimov, 2006) therefore, above 2 KHz neurones have to function co-operatively in
order to process pitch9 (Thompson, 2009, p. 115). This is possibly why humans prefer to hear
sounds below 2 KHz in frequency (i.e. anything under the top octave of the piano) when
listening to music and only use the higher frequencies for tumbrel and harmonic functions.
Figure 3: The range of piano and other instruments with the pitches above the processing limits of individual neurone indicated in Red. Pitches that approach the critical band indicated in blue. Adapted from (Blood, 2011).
9 This co-operation is also present within the cochlea; the limit is 1000 Hz (Thompson, 2009, p. 115).
There is also evidence to suggest that the bottom octaves of our musical
instruments are infringing on a critical band that appears to limit the ability of pitch
processing within the brain (Hodges, 2011, p. 112). This is the reason why close
intervals (such as thirds and fourths) sound dissonant and ‘murky’ ; the frequency
difference is smaller than the ear can comfortably detect without the two notes
interfering with one another. This is the same reason why in Bach chorales the bass and
tenor parts are kept at least a fifth (although preferably more than an octave) apart.
Only if the parts are significantly high (above D3) such as in the alto and soprano lines
can the interval between them be extremely close on the stave. However, in terms of
frequency they are not close due to the increase in the perception of pitch being on a
logarithmic scale. The limit of the critical bandwidth has been calculated to be around
20 Hz because pitch intervals increase in a logarithmic pattern a lower third on the
piano will be, in terms of actual frequency, closer to each other than the same interval
higher up the piano. For example, the two intervals shown below as part of the Bach
Chorale have a frequency difference of more than 20 Hz and are thus, for the sake of
argument, consonant. However if we were to adjust beat three of the second bar of the
extract and drop the tenor line down an octave so that the interval is between E2 a t
82.41 Hz and G2 at 98 Hz we would produce a frequency difference of 15.59 Hz. This is
within the critical bandwidth limit and the tones would start to produce an interference
pattern resulting in a sound perceived by the human auditory system as a roughness or
‘dissonance’.
Figure 4: Figure 7 the closing bars of Chorale 322.
D3 = 146.83 Hz. F#3 = 185 Hz = 38.17 Hz. G4 = 392 Hz A4 = 440 Hz = 48 Hz.
Musical cognition employs a diverse set of cognitive mechanisms in order to
accomplish the challenging task of auditory processing. The brain shows evidence of
developing a unique form of biological clock (dubbed the interval clock) that uses three areas
of the brain (basal ganglia, substantia nigra and the frontal lobes) in order to calculate the
temporal points (and therefore rhythm) that music exploits (Jordain, 1997, p. 148). Other
mechanisms such as the detection of tiny deviations in arrival time of a sound signal
contribute to our perception of an objects position in the external environment. The brain
must have developed a sufficiently frequent frame rate that is much larger than the frame rate
required for visual detection that is around 60 Hz (Deering, 1998). The frame rate of the ear
must be able to deal with deviations of less than 5 milliseconds or 200 Hz in order to build a
spatial map of the surrounding environment (Hodges, 2011, p. 96).
The theory that every part of the brain has its own unique function, otherwise known
as the modulatory hypothesis is lacking in evidence (Honing, 2011, p. 41). The conclusions
throughout this paper suggest that the brain is in fact organised in a resource-sharing
hypothesis where cognitive mechanisms interfere and overlap with one another processing
information in parallel and at different speeds. Substantial evidence for this conclusion
originates from the experiences of synesthetic individuals10 who regularly experience
interfering sensory modalities the second most common of which are auditory (after visual).
The reason for the order may be down to cognitive priorities; consciousness is composed
10 For a general but reasonably comprehensive overview of synaesthesia see ‘The Frog Who Croaked Blue: Synaesthesia and the Mixing of the Senses’ by Jamie Ward (2010).
mostly of visual data (51%), then auditory (39%) (Norretranders, 1999, p. 143) followed by
the other senses such as taste and small (that incidentally, are very often confused). This does
not account for the fact that 55% of individuals, for reasons unknown, have vision as their
dominant modularity with audition being the smallest at only 15% (Boothman, 2001, p. 119).
We are therefore inclined to perceive events using visual cognitive mechanisms even when
vision is not the sense we are employing in order to perceive the original stimulus.
The cortex of the brain also splits into two sections or hemispheres that process
certain types of neurological information more efficiently than the opposing hemisphere.
These differences (outlined in the table below) are not always clear-cut and there is a huge
range of variation between different individuals such as weather the individual is right or left
handed11.
Left Hemisphere Right Hemisphere
Verbal
Symbols
Analytic
Intellect
Successive
Serial
Convergent
Realistic
Rational
Abstract
Objective
Realistic
Visuo-spatial Images Holistic
Intuition Simultaneous
Parallel
Divergent Impulsive
Concrete Subjective
Metaphorical
Table adapted from Right Hand, Left Hand by Chris McManus (McManus, 2003, p. 243)
The range of musical experiences and tasks are so eclectic and diverse (for example
musical analysis would principally occupy the left side of the brain whereas free
improvisation would occupy the right) that music is one of the most cognitively demanding
activities. Musical processing has to straddle the two hemispheres, produce a cumulatively
11 For an in-depth discussion of the urban myth that has developed around this issue thanks to media hype among other things, see 50 Great Myths of Popular Psychology by Scott O. Lil lenfeld. (Lil lenfeld, 2009, p. 25).
cooperative outcome depending on the skill. Music can also increase the communication
between the opposing hemispheres (Levitin, 2008; Pochmursky, 2009). Music is a
neurologically holistic practice and the brain is highly adapted for the perception, creation
and recognition of musical features from a very young age because the perception of sound is
so vital to the survival of the perceiving organism thanks to its consistently omnidirectional
and long range properties (Rees, 2010, p. 1). Different auditory information is also separated
according to a specific hemisphere with language principally processed in the left-brain
(therefore perceived by the right ear) and music in the right (through the left ear). (McManus,
2003, p. 168)
A neuroscientific explanation of musical illusions is therefore decades, if not centuries
away. Due to the convoluted, complicated, and serendipitous manner of processing it is
inherently difficult to draw precise conclusions at this time.
Music Psychology
Ruling out a neuroscientific explanation of musical illusions denotes the adoption of a
more psychological perspective. Historically, the study of how the brain interprets music and
other auditory stimulation has been very interdisciplinary with connections with such diverse
fields as anthropology, physics, and biology. We are now in a unique position to collect this
body of knowledge within the explicit field of music psychology and apply it not only in a
therapeutic capacity such as curing tinnitus or diagnosing amusia but also to situations in the
arts, entertainment, and engineering industries. There is now a blurred line as to what
contributions can come from the study of music psychology. Recently, everything from
analysing the way that a car door sounds when it is shut (Etienne, 2008) to ensuring that
individuals feel comfortable when on the phone by playing music or even artificial static
interference or ‘comfort noise’ (Qian, 2006) has received a prominent standing from
academia and generous grants for further research. Accordingly, music psychology has
become a respected discipline that takes a much broader and varied outlook than just what
happens under laboratory listening conditions. At the outset of music psychological research
there was a great emphasis on sine waves or ‘pure tones’ leaving some important areas
completely unresearched (Plomp, 2001, p. 2). This has interesting parallels with the research
and application of musical illusions in that in order to apply the research effectively I will be
attempting to apply research conducted using pure sine tones to sonically complex tones
generated using instruments. The brain treats each type of tone differently as a complex tone
is, in reality a fundamental ‘pure’ tone accompanied by multiple other tones of varying
intensity (loudness) and pitch contributing to the tumbrel quality of the tone.
Jean Fourier (1768-1830) discovered that any periodic wave could be described
as the sum of its individual sine waves, where each wave is described by its frequency ,
amplitude, and initial phase (Taube, 2004, p. 309). Essentially this means that what we
wperceive as timbre is, in truth, just extra sine waves in the form of harmonics. Timbre
is, at a basic level, a result of the cognitive grouping of overtones (Levitin, 2008, p. 43). A
cathedral pipe organ functions in precisely this way, by adding more frequencies to the
fundamental by opening and closing extra pipes (using stops) one can control the
perception of the fundamental frequency and make it sound like bells (lots of
indeterminate harmonics) or a clarinet (few harmonics, strong fundamental). The brain
does not separate the overtones but integrates them into the sound perception. It is a
phenomenal development that we are able to segregate sounds from separate sources,
positions and timbres but overtones are usually largely indecipherable from the
fundamental frequency. (Thompson, 2009, p. 48) This may explain why the musical
illusions that Diana Deutsch has discovered are demonstrated as pure sine tones within
her research (Deutsch, 1995)12. Musical illusions are delicate entities, are prone to
issues of correct perception, for this reason they require an almost experimental and
controlled performance on instruments that produce timbres without many harmonics.
One can find out such information by conducting a spectral analysis of a tone from any
instrument for a given time. For instruments such, percussion, and drums the number of
harmonics produced is chaotic and seemingly random. The sheer number of complex
intervals within the harmonic series produces an almost indefinite sense of pitch. The
generally short periods of percussion instruments (including the piano) means that
most of the harmonics produced decay quickly. Whereas instruments such as the violin
contain around nine audible harmonics and are therefore much more likely to cause
auditory interference (Hodges, 2011, p. 87).
Figure 5: Fourier analysis of the piano note E4 showing overtones (harmonics) at 330, 990, 1320 and 1980 Hz. Extracted
fron (Alm, 2002, p. 459).
A poor choice of instrument would be church bells that produce a much more chaotic spectral
profile that has the overall effect of blurring the fundamental pitch:
12 Deutsch does however attempt to demonstrate their effects when performed with instruments, specifically bells but l imits this to only i l lusion (track 20) (Deutsch D. , 2003).
Figure 6: Spectral analysis of Church Bells from (Oancea, 2011).
Level -18 0 -10 -17 -34 -9 -16 -13 -23 -9 -26 -28 -29 Table 2: Breakdown of church bell analysis showing 13 (electronically) distinguishable harmonics most of whic h are not discrete notes adapted from (Oancea, 2011). The loudest overtone is an octave above the note we perceive.
The amazing data shown above demonstrates that the presence of the fundamental in
complex tones is not vital for the missing fundamental to be perceived. This phenomenon of the
missing fundamental has been observed and noted anecdotally by telephone engineers. They
witnessed first-hand these paradoxical and bizarre facts years before it acquired academic
interest due to their knowledge of the normal human range of speaking. This typically falls
between 85 – 180 Hz for a male and 165 – 255 Hz for a female (Titze, 1994, p. 188) (Orlikoff &
Baken, 2000, p. 177). Telephones operate however, on a frequency band between 300 and 3,400
Hz. When on the telephone our brain constructs the fundamental frequency from the overtone
series of the speaker’s voice and creates an illusionary fundamental frequency at the accurate
pitch of the original speaker. This has also been demonstrated empirically with musical tones on
inexpensive speakers incapable of reproducing low sonorities. (Hodges, 2011, p. 116).
Auditory and Visual processing – similarities and differences.
There are far fewer types of auditory illusions than there are types of visual illusion.
The auditory system has a larger responsibility for the survival of the organism that perceives
the auditory information; you can normally hear a potential threat before you can see it (Rees,
2010, p. 1). Consequently, there is a larger evolutionary bounty on the cognitive apparatus for
the processing and correct function of auditory processes than that of the visual modularity
from a purely evolutionary perspective. Researchers are beginning to discover that the
‘perfect’ image that we perceive with our eyes is actually largely an illusion in itself. Only
around fifteen degrees of our visual field is in focus at any one time resulting in the need for
frequent saccades when reading (Dehaene, 2009, p. 13). Further evidence for this hypothesis
originates from a developmental perspective; it takes our sophisticated visual cortex up to six
months after birth to reach that of an adult in terms of depth perception, eye control, and
experiencing vision in full colour. (Schwartz, 2004; Algoe, 2009). This is highly contrasted
with the acoustical stimulation that a foetus perceives before birth. The foetus’ ear starts
operating typically on the forty-fifth day of gestation with the cochlea beginning to process
sounds at around twenty weeks. Reaching full adult size at twenty five weeks and beginning
to process sound (but is unable to discriminate words and music or develop a sense spatial
awareness) at around eighteen to twenty weeks thus giving it a distinct cognitive advantage
over the other senses especially vision (Hallam, 2008, p. 220). Therefore, the modularity of
auditory perception has more than a seven-month advantage on the eye despite all senses
functioning fully before birth (Barenboim, 2006). The auditory information primarily consists
of internal noises (breathing, heat beat and body movements) but the most prominent is the
mother’s voice (Richards DS, 1992) aided by strong conduction on account of direct bone
conduction through the skull (Sohmer H, 2001). External auditory information is both more
diverse and carries much more information relative to corresponding senses at this time
(Hallam, 2008, p. 221).
At the other end of the life cycle, it has been hypothesised that hearing is the last
sense that we lose awareness of (Sission, 1990). Hearing is therefore our most durable and
accomplished method of perceiving the world around us and despite visions ability to
blockade perception hearing is our most pervasive sense (Hodges, 2011, p. 109).
The ideas that underlie the basics of music psychology have not changed substantially
over the last one hundred years13. Certainly new fields have been introduced (such as cognitive
psychology and neuroscience) but the basis has remained the same. Most of the basic theories
about musical perception came from research conducted in the much larger field of visual
perception and Gestalt psychology14.
Gestalt psychologists established a number of rules that apply equally well to visual and
auditory perception. Gestalt (literally ‘whole’) is concerned with finding the ‘big picture’ of how
the mind organizes smaller stimuli into larger perceptual experiences (Honing, 2011, p. 130).
Examples from visual perception where the brain has to interpret the sensory information can
be seen in infamous visual illusions such as the ‘Necker cube’, ‘Devils tuning fork’ or the
‘Penrose Triangle’ all of which exploit the brains ability to change its processing to
accommodate multiple perspectives for the perception of a stimulus based on contradictory or
paradoxical information15.
13 Most of the music psychology literature of the 20 th century was largely musical physics. One of the earliest books on music psychology is Genza Revez’s 1946 book Introduction to the Psychology of Music through which he addresses mainly the physical and acoustical issues of music psychology. 14 For a comprehensive overview of modern visual perception theory, see The Vision Revolution by Mark
Changizi. 15 More visual i llusion explanations can be found at this wonderful website maintained by Prof. Michael Bach at www.michaelbach.de
Figure 7: The Devils Tuning Fork, Penrose Triangle, and Necker Cube respectively.
Other perceptual curiosities also exist in other senses that demonstrate that perceptual
processes may be linked to one another from within the brain. Repeating the words ‘say, say,
say’ will lead to the perception of saying ‘ace’. Even illusions of touch can be demonstrated by
the ‘cutaneous rabbit’ effect whereupon a series of taps in three specific areas on the arm of a
blindfolded participant feel like an equally spaced sequence of taps. (Blackmore, 2005, p. 39).
These phenomena are the result of the brain building the illusion of consciousness from
incomplete data. The mind (and thus ourselves) does not witness reality but rather creates it.
The brain has become so proficient at creating our world that people in environments of
sensory deprivation hallucinate and create vivid experiences simply because their brains have
nothing to process and so they create their own perceptual experiences that keep their minds
active (Butler, 1998, pp. 14 - 27).
There are a number of laws that we can apply from the findings of Gestalt psychology to
the findings on musical illusions and perception16.
Proximity
This law simply states that sound sources will be treated as the same source if they are
in close proximity to one another. This proximity can be in any musical dimension as long as it is
perceptible to a listener.
Similarity
16 The following list is an adaption of the chapter on Gestalt psychology in (Hodges, 2011, pp. 130 - 132).
Musical phrases that are similar in quality are more likely to be grouped together than
dissimilar phrases.
Common Direction
Music that is moving in a common direction is more likely to be grouped together as a
perceptual unit than music moving obliquely.
Figure-ground Relationship
This law states that when we perceive something we focus and enhance the features
that stand out allowing other details to fade into the background of awareness. A tone, if
sustained for significant duration (several minutes) would eventually regress into the
background of consciousness if new, important information is perceived.
Law of Prägnanz
Patterns are organized in the simplest way possible. If an audience encounters an
unfamiliar piece of music or sound they will impose order on it in a predictable manner. This
applies to all aspects of music most notably meter perception. (Hodges, 2011, p. 130)
Closure
The perception of an ending (perfect cadence) at the conclusion of a musical phrase is
cognitively appealing. In terms of melody, a return to the tonic functions using the same
preferences.
Research on the topic of auditory and musical illusions is not particularly prevalent in
current literature. Only a handful of chapters (within books related to broader topics such as
perception) exist on how and why and how they function17. This is in great contrast to the
17 See (Deutch 2003) for the most comprehensive treatment.
literature on visual illusions18. Both types of illusion actually have frequent and reasonably
strong connections to each other suggesting that they may be neurologically linked (Plomp,
2001, p. 43; Shepard, 1990, p. 147). The difference in academic interest might be attributable
to the perceptual qualities of each type of illusion (Jordain, 1997, p. 140). Musical illusions
are much more subjective and depend on specifics such as location and positioning, the type
of environment the perceiver is currently situated around, and the type of listening device that
they are using along with other such parameters that make them difficult to encounter in
everyday experience; one cannot point to an auditory illusion. Visual illusions, on the other
hand, do not have all these caveats and are therefore much more accessible, instant and thus
observable than auditory illusions.
The brain receives eleven million bits of data per second and only a tiny fraction of
these (100,000, the same amount as for the sense of smell) are attributed to auditory
perception the vast majority (10,000,000) are purely visual bits of information (Schmidt,
1989). Because of their elusive qualities, auditory illusions are also more personal and
therefore more powerful for the individual experiencing them. An auditory illusion can take
the mild ‘reality checks’ that a visual illusion might provide to an entirely new level. One of
the main aesthetic qualities of music is to saturate the mind of the perceiver (Jordain, 1997, p.
xi). When an auditory illusion saturates the individual’s sensory systems this can highly
disconcerting; an individual can look away from a visual illusion whenever they choose to
whereas an auditory illusion forces you into observing the phenomena without choice thereby
providing a dramatic and memorable experience. As Daniel Barenboim states in his lecture
‘The Neglected Sense’ ‘we cannot shut our ears” (Barenboim, 2006). Further evidence that
the role of hearing is the underdog of perception can be witnessed by highlighting flaws that
occur frequently in human vision. The depth of awareness that we commonly experience
18 See (Seckel, Al, 2004, 2006, 2007, and 2010).
when visualising the world is in fact a complete illusion. We are only aware of a tiny fraction
of visual information at any one time because of our forward facing eyes and small focus area
on the back of the retina meaning that we perceive far less than we imagine. The brain creates
our world for us from very limited information (Lawrie, 2012) but we have the illusion of a
rich visual experience purely because we have never experienced any alternatives.
The Musical Applications of Psychological Research
I am not the first composer to consider using illusions in a musical composition.
Artists such as Queen19, Pink Floyd20, Bjork21, Wilco22, and Gorillaz23 along with several
film composers such as Hanz Zimmer and James Newton Howard24. They have all used the
Shepard tone illusion where a phrase seems to endlessly increase or decrease in pitch as part
of their compositional output. Even classical composers such as J.S. Bach composed pieces
exploiting the properties of musical illusions as can be seen in an ‘endlessly rising canon’
from Canon a 2, per tonnes from Bach’s Musical Offering (Hofstadter, 2000, p. 712). The
technological developments of music along with discoveries in the field of psychology and
neuroscience should allow musical illusions a larger role within musical works. The illusions
themselves could begin to define the features of the work rather than be the quirky effects
that they function as within current compositions. In the second half of this paper, I shall
employ a specific set of illusions as the basis for two compositions and derive most of the
material from the application of these illusions combining them with more ‘traditional’
musical material to make a compelling musical-conceptual experience.
Most of the musical illusions will look at exploit the auditory dimension of pitch. The
range in pitch that humans can typically experience is roughly between 20 to 20,000 hertz
(Backus, 1977) but this varies considerably with age and pervious musical experiences and
even on the perceptual and conditioning effects of music (Deutsch D. , 1998, p. 236). Under
19 The album A Day at the Races (1976) contains two tracks, which include an ever-ascending scale. Tie your mother down and Teo Torriatte. 20 Echoes (1970). 21 The album Biophillia (2011) contains two tracks containing Shepard tones. Cosmogony (2011) features a descending choir whereas Mutual Core features an ascending effect. 22 Born Alone (2011). 23 Dare (2005) features a Shepard glissando. 24 The bat mobile engine sound is a Shepard tone that always ascends providing the perceiver with the experience that it is constantly accelerating (Jackson, 2008).
normal circumstances, our ears loose around one-half cycle per second per day from the
moment we are born (around 160 cycles per second per year) (Jordain, 1997, p. 17). There is
another way to experience pitch that is lower than twenty hertz (called infrasound) but this is
through physical sensation and physiological effects rather than with the ears. The
psychological effects of infrasound have been investigated (anecdotally) perhaps most
notably by Vic Tandy who developed a reputation for debunking paranormal myths in the
late 20th century when he experienced a ‘ghostly’ apparition whist working in a research
laboratory for a medical manufacturing firm. Tandy then experienced a ghostly apparition in
his peripheral vision that vanished when he turned to observe it properly. The experience was
later attributed to imperceptible low frequency noise emanating from a recently installed
extractor fan that activated the physiological ‘fight or flight’ response, this generated tension,
and unease and caused Vic’s eyes to vibrate in their sockets causing the ‘ghostly’ apparition
(Lyster, 2001). These infrasound vibrations have also provided an explanation for reports of
the London underground being haunted with extremely high levels around areas that are
frequently reported to bizarre experiences (Kane, 2006).
Now the experimental data and methods have caught up with curious and routine
phenomena, we are now in a position to attribute a whole host of physiological responses to
sound and music in a variety of settings (Hallam, 2008, pp. 121- 130). Composers can use
this information to benefit their compositions to make them more emotionally varied and
effective.
In order to ensure that the complete auditory experience is as strong as possible
I will be controlling the visuals and the lighting conditions within the performance hall.
Recent research suggests that when one of our sensory experiences is not being used
we increase the perception of our other senses due to the lack of distraction from the
sense that we have lost and are able to focus specifically on that particular sense
(Dachis, 2011). We do not receive any more information from the sensory deprivation
but we are able to process the remaining senses more effectively. I wish to focus the
audience on their auditory system and ensure that for the parts of the composition that
require the most concentration that there are significantly few distractions that hav e
the potential to lessen the overall effect. It is with this intention that I wish to be at the
centre of the concert program; the audience will at this point be comfortable and
reasonably oblivious to their surrounding environment after becoming habitualized to
their enviroment over the course of the concert. Thus, their senses of taste, smell and
touch senses will be in a state of comparative limbo. Removing vision from this allows
them to focus their entire consciousness on their auditory perception allowing the
illuions to have their greatest chance of impact.
The training that I have received within composition has been of a classical
nature. It is for this reason that I am limiting the type and function of musical illusions
that I will use in this paper to those that can either be notated in a score and performed
on acoustic instruments or achieved through the minimal use of music technology and
samples. The power and range of possibilities that music technology offers to the
composer today is a bewildering state of infinite possibility. Some of the illusions
themselves dictate the type of equipment that should be used in order to perceive their
correct effects such as the use of headphones on stereo field illusions . Because both of
the works that I am writing will be performed in a concert setting, I will be focusing
most of my explanations to the effects that work best through loudspeakers with the
minimum of auditory processing and a reliance on the performance score. I have
attempted to ensure that the stage and performance setting are established with the
aim of developing the best approximation of the stereo pan of headphones by ensuring
that the speakers are facing the audience directly (orientated at 5 degrees towards the
wall that is closest in proximity to each speaker. This ensures that there will be minimal
‘spillage’ between the two and that the sound will reflect off the wall and into the
respective ear more accurately. Acoustic instruments audiate in an omnipresent
manner; they are therefore much more difficult to control in a live performance setting.
Illusions of spatial setting will therefore feature only in the stereo recording and not
through the acoustic instruments25.
Applications in Compositional Practice
COLLATERAL (2012)
The first composition on the topic of cognition and psychology of music was based on
some of these ideas. Collateral has been written for two pianos and the work attempted to
mirror the functions and biases of the brain by translating the various features of both sides
into performances on the piano. The right performer would encounter a lot of opportunities
for improvisation whereas the left performer would perform the work verbatim to how I had
composed it (see fig 1). I also included several musical illusions within the work at mapped
them onto the opposing piano with the hope that the audience would process it with the
correct side of the brain in order to achieve the complete effect and experience.
25 Although a small amount of stereo pan will sti l l be attainable through the position of the quartet on the stage.
Figure 8: The same musical passage in different guises simulating 'right' and 'left' hemisphere characteristics.
Collateral was to be centred on the ideas of brain lateralisation so I chose to orchestrate for
two identical instruments each being a representation of one hemisphere of the brain. Akin to a
scientific experiment, I imposed controls on my composition by limiting the timbre and the location
of the sound sources (grand pianos are not easy to relocaste). I can therefore now focus on the
remaining attributes that form music: pitch, rhythm and volume. I shall also use this work to
demonstrate the cognitive effects of harmony, melody and musical recognition.
I specifically refer to three types of illusion within this work. All of these illusions are based
on the cognitive property of stream segregation. In order to accommodate the technique generating
illusions as musical material I had to look for an adventurous method of harmonizing the music in
order to make the illusionary sections fit against the backdrop and blend in with more conventional
musical material.
The Tritone Paradox
Originally discovered in 1986 (Deutsch D. , 1986), using computer generated tones, the
tritone paradox displays a discrepancy of relative pitch. Certain intervals appear to ascend or
descend depending on where they appear on the chord wheel.
One hypothesis of why this perception distribution exists may come from speech perception.
Differences in perception have been found in test subjects who are not from California where this
phenomenon has been examined. Frequently when a Californian subject heard a pattern as
ascending, a subject from the south of England heard the identical pattern as descending, and vice
versa (Deutsch D. , The tritone paradox: An influence of language on music perception, 1991) . Other
studies have also provided further evidence of this language dependence for perception of the
tritone paradox. But there are several complications as it appears that geographical area of
childhood upbringing can affect the way that someone perceives the tritone paradox.
The Cambiata Illusion
This illusion exploits the brains ability of stream separation by providing two distinct
‘streams’ of notes at a rapid tempo. The illusion’s effect and perception can be seen below.
As with the octave illusion, there are differing reactions to this effect with some people hearing the
above and others hearing more complex derivatives such as the following:
For right handed individuals, the pattern seems to remain with the lower pitch stream
seeming to originate from the left ear and the higher in the right. This may reflect the contralateral
arrangement of the auditory pathway with stimulation from the left ear being processed by the right
side of the brain and vice versa (McManus, 2003, p. 168). Left handed individuals are also more likely
to experience the complex perceptions rather than the typical one that most right handed
individuals experience. The possible reasons for this are not currently clear. (Deutsch D. , 2007) The
brain separates auditory information along eight specific dimensions such as pitch, timbre, location
and loudness of the sound. (Deutsch D. , 1998, pp. 313 - 321) In this instance the brain decides that
because the pitch difference between the two notes is so large (two and a half octaves) the reality is
most likely to be that there are two separate instruments and they are separated spatially with the
high tone on the left and the low tone on the right. The reason for the differences in the perception
by right and left handed individuals is not currently known and there is also no strong hypothesis for
why the brain as a whole (apart from the sense of smell) is wired in a contralateral manner (Ratey,
2003, p. 64). This illusion is can also be used with a chromatic scale and give the listener the
perception of two separate scales when they are in fact performed by the same instruments.
The Chromatic Illusion
This illusion has the same mechanism as that of the Cambiata illusion in the previous
section.
It functions as the result of gestalt proximity, similarity and simplicity and functions as the
brain cannot handle this much complex information so a shortcut is taken providing and incorrect
perception of reality (Jordain, 1997, p. 249).
ULTRA (2012)
My second composition is orchestrated for clarinet, string quartet, piano and laptop. I have
chosen this instrumentation very deliberately with the intention of matching it to a specific
outcome. For example I have used the clarinets and piano when using elements that require the use
of purer tones and the string instruments (that have more complex overtone patterns) for the more
musical elements. I have also experimented with the positioning of the instruments within the piece
and focused on developing a depth of field effect with runs of notes across the ensemble and a
heavy pan effect on the musical illusion at the end. The harmony of the work is derived from the
Nicolas Slonimsky method of octave division through the interval of a tritone (Slonimsky, 1999). This
interval has been shown to generate a strong cognitive response in western listeners and therefore
deviates musical expectations overloading the sensory areas of the brain and consequently causing it
to focus on other musical events (such as rhythm or the illusions) (Thompson, 2009, p. 50). This is a
theme that runs throughout ULTRA with the last chord serving as both a lesson in the ridiculousness
of concert etiquette and as a demonstration of musical and body language effects. The inclusion of
electrical instruments within this piece also allows me to have exact control on the frequency of
certain tones with the inclusion of electronic instruments as acoustic or analogue instruments, by
their very nature never produce pure, unfaltering tones. I also involve the facility of speech
perception within this composition. This allows me to tap into the most important feature of our
brains – the ability to communicate using words. The brain is remarkably adept at constructing
sounds from distorted auditory data so much so that illusions can be created and words perceived
where none exist simply by presenting and repeating a short fragment of a word in each ear over an
extended duration. In order not to ensure that the central focus of the piece at this point is with the
works I have kept the accompaniment to a minimum with a scarce texture of static chords moving in
a stepwise motion. The auditory illusions develop over time and therefore it is extremely important
that this section be the main focus of the work. The illusions works because our brain is rapidly
trying to make sense of the data that it is receiving. The brain recognises (due to timbre and other
qualities) that the sounds that it is perceiving are supposed to be words and then fills in the gap
according to expectation. The samples contain short sounds and since 67% of words in the English
language are formed from such short sounds and consonants the brain has a wide selection guesses
to enable it to force an interpretation. Further anecdotal evidence can be provided in the fact that
the words often match what the listener is currently, or has been thinking about such as a diet or
food. This is an example of priming and it offers further evidence that the brain is providing us with
the experience that it expects us to perceive. Indeed sometimes is can seem that music and speech
are linked in some description. Evidence of this can be seen at the start of ULTRA where the bass
clarinet mimics the tones of a sentence spoken normally but because we have experienced it under
musical terms we now perceive it as a musical phrase. It is often said that there is a link between
music and language but it has only been stated a few times (most notably by Mussorgsky in a letter
to Rimsky-Korsakoff) that music and speech are similar. Indeed, Mussorgsky believed that he could
actually compose convocation “whatever speech I hear, no matter who is speaking … my brain
immediately sets to working out a musical exposition for this speech”. (Emerson, 1999, p. 75)
There can therefore be seen to be a blurred line between speech and singing with speech taking on
a musical quality when repeated. Recent research has also discovered that the brain processes
speech faster than musical tones further suggesting that speech is now a fundamental attribute of
our brains and providing further support for the modularity hypothesis suggested earlier. So at this
point in the piece, the brain will focus more on the speech sounds than on the relatively
uninteresting chordal accompaniment provided by the string quartet.
Musical memory is also a strong feature of this composition where I attempt to induce a
similar effect to a musical hallucination at the edge of the perception threshold. I chose two well -
known pieces of piano music (since I am performing this to an audience of musicians) that most of
the audience are very likely to know (or very likely to have played) from their musical upbringings.
The pieces (Claire De Lune by Debussy and Moonlight Sonata by Beethoven) are played in each ear
respectively overlapped with ‘white noise’. It has been found that masking a stimulus by placing
wide frequency noise over it has the effect of accentuating the resulting perception as the DAP
focuses on the ‘interesting’ content and filters out the ‘uninteresting’ white noise. In order to ensure
that the perception of these pieces felt like an auditory illusion I ensured that the volume of the
pieces from both speakers did not exceed – 15db.
The Octave Illusion
This illusion appears in Collateral and features the two pianists performing an octave interval
in an antiphonal manner. The intended perceptual effect of this illusion is that each piano each
performs an Individual note at an explicit pitch. However, despite its relative simplici ty there have
been many reported perceptual results of this illusion ranging from the simple instances (such as
those above) to the complex
Explanations for this effect rely on the type of perceptual experience that the listener
generates in their mind however this type of effect relies heavily on the stereo pan and therefore
may not function too well on two pianos in a concert venue. What is more, the perception of this
illusion, as do many others depend on whether the perceiver is right or left handed and the brain
also appears to adapt the perception of tones if the listening apparatus are physically altered (i.e.
turning the headphones onto the opposite ears) (Deutsch D. , 1974(a)) (Deutsch D. , 1974(b)).
The primary reason for the effect of this illusion on our conscious experience of the sound is
that all the sounds that we perceive are grouped by the brain into ‘similar’ and ‘dissimilar’
categories. This auditory grouping helps us to separate different sound sources and build a mental
map of what is happening around us.
The Scale Illusion
The most fundamental and simple application of musical illusion can be seen in the scale
illusion. The two pianos perform a set of notes that constitute parts of a scale. The brain perceives
these as a set of serial tones from each sound source even if different instruments are used (Se e
ULTRA) (Jordain, 1997, p. 248).
This illusion is an example of the three gestalt principals of similarity, proximity and common
direction. The brain does not have the processing power in order for it to conclude that it is being
deceived it therefore invents the same scenario that it has encountered many times before; a
normal scale performed by two instruments or a four note pattern that ascends then decends.
Illusions and Speech Perception and the Mysterious melody
The opening of ULTRA contains within it a different type of illusion. Rather than create a
paradox or interesting perceptual experience the simple sentence that appears at the start of Diana
Deutsch’s ‘Musical Illusions and Paradoxes’ CD exercises the blurred line between speaking and
singing to demonstrate that the perceptual experience of each is not as mutually exclusive as some
people believe (Deutsch, 1995, pp. 19-23).
Speech is constructed of long and short sounds (vowels and consonants). These vowels, by
their very nature of being sustained require a pitch element. Combined with a reasonably strong
sense of rhythm a simple sentence can begin to sound a lot like a musical phrase.
Further paradoxes of speech are found on Phantom Words and Illusions where Deutsch
separates words into syllables and pans each to the left and right respectively at four times per
second (240 bpm). The brain cannot find any useful information from this data so it starts to invent it
and perceive words that are not truly present. Further words can also be formed by combining the
tracks from the audio CD as occurs in ULTRA due to toe timing restrains of a live performance.
One particularly striking effect is that the words that people perceive can be primed by what
they are thinking at the present time (Deutsch D. , 2003, p. 3).
The Quickening Pulse
There is a type of rhythmic illusion that I used in ULTRA that gave the impression of a
quickening pulse when in reality the tempo remained completely static but this effect can only be
realistically achieved through the application of music technology and sampling techniques.
This effect was first created by Kenneth Knowlton and Jean-Claude Rissett by superimposing
drum rhythms that have similar geometric relationships to one another (Pierce, 1983) (Risset, 1989).
This effect is very similar to an effect that one can achieve with a set of closely relating pitches.
Tones that are separated by one frequency cycle a second seem to pulsate as an interference
pattern is created and they constructively and destructively interfere with each other. This beating
effect is very pronounced with a difference of one hertz but increases in tempo and becomes less
apparent with each increase in pitch until the two tones become separated after the critical band
has been exceeded (Hodges, 2011).
The illusions that I have referred to thus far have concerned only the dimension of pitch perception.
Pitch perception is by far the most highly researched area in the entire field of music cogniti on. Here,
I briefly demonstrate that illusions can also be created with rhythm but that our categorisation of
rhythm is such that perceptual experiences are so varied that to call any rhythm illusionary seems
bizarre (Thaut, 2005, p. 4). We experience musical illusions in most pieces of music the classic being
the hemiola or polyrhythm that forces us to perceive separate rhythms when there is really only one
complex rhythm. The perceptual mechanisms, for example stream segregation for detecting rhythm
appear to be superior to pitch perception perhaps due to their evolutionary importance such that
there are far fewer illusions that can be created. For example, practically everyone with even limited
musical training can perceive and identify a hemiola with a 3:2 relationship. The brain separates
these immediately into two separate streams rather than combining them into one complex melody.
As we have seen this is unlike pitch perception which can struggle to separate out multip le streams
of information due to overtones, sensory overload or other cognitive phenomena.
The Shepard Tone
This illusion has a direct visual to the visual illusion generated by the
rotation of a barber’s pole. When the pole is spun it creates the illusion of the
ribbon that is painted on the surface moving either horizontally or vertically. This
precisely demonstrates the effect in terms of music but with pitches instead of a
rhythm. This is potentially the most frequent musical device or illusion in
compositional terms and it has been found in music spanning most of musical
history from Bach (Hofstadter, 2000) to Muse26.
The illusion was first discovered by Robert Shepard at Bell Laboratory in 1958 where he
attempted to build an illusion to accompany his version of the Esher impossible staircase . The
illusion works best using computer generated sounds due to the lack of interfering overtones but
there have been some versions using musical notation perhaps the most famous being the closing
chapter from Douglas Hofstadter’s Godel, Esher, Bach using the computer program SMUT.
(Hofstadter, 2000, p. 713).
Shepard published his new illusion in a paper in 1964 (Shepard, 1964) demonstrating how
this illusion an ‘breakdown transivity in judgements of relative pitch’ simply by presenting the same
notes over and over again with the peripheral parts modulated to procure a smooth transiti on.
26 ‘Ruled by Secrecy’ 2003. 4:40 – end.
Figure 9: The Shepard Tone Illusion from ULTRA. Adapted from (Hofstadter, 2000, p. 713)
Conclusions
It can be seen that there are a wide variety of useful applications of musical illusions and
other research within the field of modern composition. Illusions in general also appear to use the
same (or very similar) cognitive mechanisms in order to generate the conscious experience of
perception. Consciousness is still a widely debated and controversial topic of significant interest and
further research is required into the processes behind the conscious experience before we can
comprehensively examine its flaws and quirks such as visual and auditory illusions.
Bibliography Algoe, A. (2009, March). a. Retrieved March 18th, 2012, from Hub Pages: