-
Music Processing in Deaf Adults
with Cochlear Implants
by
Mathieu R. Saindon
A thesis submitted in conformity with the requirements for the
degree of Master of Arts
Graduate Department of Psychology University of Toronto
© Copyright by Mathieu R. Saindon 2010
-
ii
Music Processing in Deaf Adults with Cochlear Implants
Mathieu R. Saindon
Master of Arts
Graduate Department of Psychology University of Toronto
2010
Abstract
Cochlear implants (CIs) provide coarse representations of pitch,
which are adequate for speech
but not for music. Despite increasing interest in music
processing by CI users, the available
information is fragmentary. The present experiment attempted to
fill this void by conducting a
comprehensive assessment of music processing in adult CI users.
CI users (n =6) and normally
hearing (NH) controls (n = 12) were tested on several tasks
involving melody and rhythm
perception, recognition of familiar music, and emotion of
recognition in speech and music. CI
performance was substantially poorer than NH performance and at
chance levels on pitch
processing tasks. Performance was highly variable, however, with
one individual achieving NH
performance levels on some tasks, probably because of
low-frequency residual hearing in his
unimplanted ear. Future research with a larger sample of CI
users can shed light on factors
associated with good and poor music processing in this
population.
-
iii
Acknowledgments
This thesis would have not been possible without the constant
efforts, guidance and dedication of
my supervisors Dr. Sandra Trehub and Dr. Glenn Schellenberg. I
am very grateful for all of their
help with this research project.
I would also like to thank my parents and sister for their
long-distance support, and my lovely
wife Lauren for baking all of those muffins.
Lastly, I would like to thank the health professionals and
patients of the Sunnybrook Cochlear
Implant Program. Without them, this project would not have been
possible.
-
iv
Table of Contents
Acknowledgments
..........................................................................................................................
iii
Table of Contents
...........................................................................................................................
iv
List of Tables
.................................................................................................................................
vi
List of Figures
...............................................................................................................................
vii
List of Appendices
.......................................................................................................................
viii
1 Introduction
................................................................................................................................
1
2 Method
.......................................................................................................................................
4
2.1 Participants
..........................................................................................................................
4
2.2 Apparatus
............................................................................................................................
6
2.3 Test Battery
.........................................................................................................................
6
2.3.1 Metric task
..............................................................................................................
7
2.3.2 Rhythm task
............................................................................................................
7
2.3.3 Distorted Tunes Test
...............................................................................................
8
2.3.4 Musical emotion test
...............................................................................................
8
2.3.5 Diagnostic Analysis of Nonverbal Accuracy 2
....................................................... 9
2.3.6 Open-set word recognition
......................................................................................
9
2.3.7 CAMP test
...............................................................................................................
9
2.3.8 Familiar music task
...............................................................................................
10
2.3.9 Pitch- and interval matching
.................................................................................
11
2.4 Procedure
..........................................................................................................................
11
3 Results and Discussion
.............................................................................................................
12
3.1 Open-Set Word Recognition
.............................................................................................
12
3.2 CAMP
...............................................................................................................................
12
-
v
3.3 Distorted Tunes Test
.........................................................................................................
15
3.4 Familiar Music Task
.........................................................................................................
17
3.5 Metric Task & Modified MBEA Rhythm Task
................................................................
20
3.6 Music Emotion & DANVA2
............................................................................................
22
3.7 Pitch- and Interval-Matching Task
...................................................................................
25
3.8 Conclusion
........................................................................................................................
27
References
.....................................................................................................................................
29
Appendix
.......................................................................................................................................
36
-
vi
List of Tables
Table 1. Participant
Characteristics……………………………………………………………….5
Table 2. List of CVC Words………………………………………………………………………9
Table 3. Music Emotion Arousal
Scores……………………………………………………...…24
-
vii
List of Figures
Figure 1. Syllable (CVC)
Recognition…………………………………………………………...13
Figure 2. CAMP Pitch Threshold
(semitones)………………………………….....……………..13
Figure 3. CAMP Melody Recognition (Percent
Correct)….…………………………………….14
Figure 4. CAMP Timbre Recognition (Percent
Correct)..……………………………………….14
Figure 5. Distorted Tunes Test…………………………….……………………………………..16
Figure 6. Familiar Music – No-Rhythm Condition (Percent
Correct)…………………………...17
Figure 7. Familiar Music – Melody Condition (Percent
Correct)………………………………..18
Figure 8. Familiar Music – Instrumental Condition (Percent
Correct)…………………………..19
Figure 9. Metric Task…………………………………...………………………………………..21
Figure 10. Modified MBEA Rhythm Task………………………………………………………22
Figure 11. Music Emotion Task.…………………………………………………………………24
Figure 12. DANVA 2: Adult Vocal Emotion
Task…...…………………………………………25
Figure 13. Average Deviations in Pitch
(semitones)…….………………………………………26
Figure 14. Deviations in Interval Matching
(semitones)……………...…………………………26
-
viii
List of Appendices
Appendix A. Music and Cochlear Implants
Questionnaire…………………………………...…36
Appendix B. Music Background Information Questionnaire
(Adults)…………………………..42
Appendix C. Music and Cochlear Implants
Interview…………………………………………...44
Appendix D. Semi-Structured
Interview……………………………………………………..….45
-
1
1 Introduction
A cochlear implant (CI) is a prosthetic device designed to
provide hearing sensations to
deaf individuals. Unquestionably, it is the most successful
neural prosthesis to date, as viewed by
the number of individuals who have received it worldwide and
derived great benefit from it
(Wilson, 2004). Its external microphone and signal processor
receive incoming sound, transform
it into an electrical signal, and extract features that are
important for speech perception. This
information is then transmitted to electrodes implanted in the
cochlea, and, in turn, to the
auditory nerve.
Modern devices provide relatively coarse representations of
spectral information, which
are adequate for perceiving speech in ideal listening conditions
(Shannon, Zeng, Kamath,
Wygonski & Ekelid, 1995; Wilson, 2000), but they are
inadequate for perceiving speech in noise
(Fetterman & Domico, 2002; Firszt et al., 2004), identifying
emotion from speech prosody
(Hopyan-Misakyan, Gordon, Dennis & Papsin, 2009; Meister,
Landwehr, Pyschny, Walger &
Von Wedel, 2009), differentiating one speaker from another
(Meister et al., 2009), identifying
musical timbres or instruments (McDermott & Looi, 2004), and
recognizing melodies from pitch
cues alone (Kang et al., 2009; Kong, Cruz, Ackland-Jones &
Zeng, 2004).
Music perception is especially challenging for CI users. Coding
strategies in implant
processors extract the temporal envelope, discarding the
temporal fine structure that is critical for
music perception (Galvin, Fu & Shannon, 2009). Consequently,
the music perceived by CI users
is considerably degraded in sound quality and detail, especially
as it pertains to pitch patterning.
In fact, implant users often describe music as unpleasant,
mechanical, and difficult to follow
(Gfeller, Christ, Knutson, Woodworth, Witt & DeBus, 1998;
Gfeller, Witt, Stordahl, Mehr &
Woodworth, 2000; Lassaletta et al., 2007). It comes as no
surprise, then, that postlingually
deafened adult CI users, who had access to rich auditory
representations of music before their
hearing loss, are often disappointed with music heard via their
implant (Gfeller, Christ, Knutson,
Witt, Murray & Tyler, 2000; Lassaletta et al., 2007; Looi
& She, 2010; Veekmans, Ressel,
Mueller, Vischer & Brockmeier, 2009). This is unfortunate
because music is an important source
of pleasure for many, if not most, hearing individuals (Laukka,
2006). Even for postlingually
-
2
deafened implant users, quality of music perception is
associated positively with quality-of-life
ratings (Lassaletta et al., 2007).
As noted, limited temporal fine structure or spectral detail
provides limited access to
pitch patterning. Cooper, Tobey, and Loizou (2008) used a test
battery designed for the diagnosis
of amusia, or tone deafness, in individuals with normal
audiological profiles. They found that CI
users failed to discriminate two melodies that differed in pitch
patterning even when the
difference involved a change in pitch contour or key. In that
sense, CI users performed much like
amusic individuals who are typically deficient in the perception
of pitch patterns but not
temporal patterns (Foxton, Nandy & Griffiths, 2006). In
other research, CI users have exhibited
difficulty determining which of two sounds is higher in pitch
(also referred to as pitch ranking –
see Kang et al., 2009; Looi, McDermott, McKay & Hickson,
2004, 2008), detecting the direction
(higher or lower) of a pitch change in a melody (Gfeller et al.,
2007; Leal et al., 2003), and
differentiating melodies in the absence of rhythmic cues (Kang
et al., 2009; Kong et al., 2004).
In the context of these pitch perception difficulties, it is not
surprising to find deficient pitch
production as well. For example, child CI users preserve the
rhythms but not the pitch contours
(i.e., patterns of rising and falling pitches) when they sing
familiar songs (Nakata, Trehub,
Mitani & Kanda, 2006; Xu et al., 2009). Although this
pattern is mirrored, to some extent, in the
song production of amusic individuals, some individuals with
severe pitch perception deficits
manage to produce accurate contours and intervals when singing
familiar songs with words,
which reveals an unexpected dissociation between perception and
action (Dalla Bella, Giguère &
Peretz, 2009).
By contrast, tempo and rhythm perception in CI users are
reportedly comparable to
normally hearing (NH) listeners except when the stimuli or tasks
are complex (Cooper et al.,
2008; Gfeller, Woodworth, Robin, Witt & Knutson, 1997; Kong
et al., 2004). Although we have
learned much in recent years about the music perception skills
of CI users, much remains to be
learned. For example, the perceptual demands of differentiating
simple rhythm or pitch patterns
differ drastically from the demands of perceiving conventional
music on the radio, on iPods, or
in concert halls. Rhythm, pitch, and timbre are typically
blended into a coherent whole.
Discriminating two rhythms in isolation does not mean that a CI
user would be able to hear a
guitar solo when it is accompanined by a drum kit, bass, guitar,
and vocals. He or she might also
be unable to pick out the recurring cello melody in a Beethoven
symphony. In short, there is little
-
3
understanding of CI users’ ability to perceive music as they
might hear it on a recording or at a
concert.
In addition to providing pleasure and contributing to quality of
life, music perception
skills underlie the perception of emotion in speech as well as
music (Juslin & Laukka, 2003).
Emotion in speech is conveyed primarily by musically relevant
cues such as loudness, tempo or
rate, rhythm, pitch height, pitch range, and pitch contours. For
example, expressions of anger in
speech and music typically involve rapid tempo and increased
amplitude or loudness in contrast
to expressions of sadness, which typically involve slow tempo,
low pitch, and decreased
loudness. Although word recognition is obviously crucial for
successful verbal communication,
it is difficult to discern a speaker’s true emotions and
communicative intentions without access to
paralinguistic and prosodic cues. To date, however, there has
been little research on CI users’
perception of emotion in speech and none on their perception of
emotion in music.
The goal of the present study was to provide a comprehensive
assessment of the music
perception skills of adult CI users who became deaf
postlingually. The perception of rhythm and
pitch perception was assessed. The perception of emotion
conveyed through speech and music
and pitch production were also assessed. Rhythm perception was
assessed in the context of
simple rhythmic patterns as well as melodies with accompaniment.
Adding accompaniment to a
simple rhythm test used previously with adult CI users (Cooper
et al., 2008) made it possible to
determine whether “normal” rhythm perception skills remained
evident in ecologically valid
musical contexts. Melody perception was assessed by means of
tasks that required comparisons
of the musical input with long-term representations of music.
The perception of emotion in
speech was assessed with a task that has been used with child CI
users (Hopyan-Misakyan et al.,
2009). Although child CI users were unsuccessful at
differentiating vocal emotions, it is possible
that adult CI users, by virtue of their previous access to
acoustic information and their greater
understanding of communicative conventions, might be more
successful than children at this
task. Finally, we tested open-set word recognition, using
monosyllabic consonant-vowel-
consonant words, as a check on CI users’ use of bottom-up cues
in speech.
Large individual differences are pervasive in CI outcomes.
Factors influencing outcomes
among postlingually deafened adults include duration of
near-total deafness (i.e., little or no
benefit from hearing aids) before implantation, with shorter
durations having more favorable
-
4
outcomes (Van Dijk, Van Olphen, Langereis, Mens, Brokx &
Smoorenburg, 1999); cognitive
abilities (Pisoni & Cleary, 2004); integrity of the auditory
nerve and central auditory system
(Hartman & Kral, 2004; Leake & Rebscher, 2004); and
relevant experience or training (Fu,
Nogaki, & Galvin, 2005; Galvin, Fu & Nogaki, 2007).
Adults with residual hearing immediately
prior to implantation perform better on subsequent recognition
of speech and environmental
sounds than those without usable residual hearing even though
implantation destroys the residual
hearing (Van Dijk et al., 1999). Moreover, CI users with music
training in high school, college,
or later exhibit better music perception (Gfeller et al., 2008).
Based on these findings and our
own specific goals, we designed a questionnaire that could
potentially shed light on individual
differences in performance. Information was solicited about
education, history of hearing loss
and implantation, implant characteristics, music listening and
music-making habits, and music
training. We expected CI users to perform poorly compared to NH
listeners except on the test of
simple rhythm discrimination. We also expected performance to be
affected by duration of
deafness before implantation, musical exposure and training, and
residual hearing, if any, in the
unimplanted ear. Finally, we expected CI users to perform better
on musical materials that were
highly familiar to them than on those that were less familiar or
unfamiliar.
2 Method
2.1 Participants
The target participants were adult CI users (n = 6) 46-76 years
of age (M = 62.2, SD = 13.0; see
Table 1) who were recruited from the Cochlear Implant Program of
Sunnybrook hospital in
Toronto. All of them were postlingually deafened, they
communicated solely by auditory-oral
means, and they expressed some interest in music. Additionally,
they all reported progressive
hearing losses that were gradual, except for one participant.
Although she experienced
substantial hearing loss when she was very young, her bilateral
hearing aids were very helpful
until 6 years ago when she experienced a precipitous loss of
most of her residual hearing. One
participant used a hearing aid in his unimplanted ear to amplify
his residual hearing selectively at
500 and 250 Hz (90 and 70 dB thresholds, respectively). With
respect to musical background,
three CI users had taken music lessons in the past, but only two
were still playing music.
-
Table 1. Participant Characteristics
Participant M/F Age Device(s) Type of CI
Hearing loss
onset (age)
Progressive loss
(yes/no)
Hearing aid use (years)
Implant use
(years)
Music lessons (years)
Current instrument
Weekly music listening (hours)
CI-1 F 47 2 CIs Advanced
Bionics 1 yes
sudden 40 6 0 No 7 – 10
CI-2 M 46 CI + HA Cochlear 5 yes
gradual 35 5 5 Yes 10 or more
CI-3 F 67 CI Advanced Bionics 57 yes
sudden 5 5 7 No 4 – 7
CI-4 F 74 CI Advanced Bionics 35 yes
gradual 30 4 23 Yes 1 – 4
CI-5 M 76 CI +HA Med-El 58 yes gradual 12 2 0 No 1 – 4
CI-6 F 63 2 CIs Cochlear 10 yes
gradual 35 17 0 No 1 – 4
5
-
6
The control group consisted of normally hearing (NH) listeners
(n = 12) 19-58 years of
age (M = 29.0, SD = 13.8) with no personal or family history of
hearing problems. A few
participants in the control group had received music lessons as
children, but only two had
substantial musical training. One of these was a professional
musician.
2.2 Apparatus
Testing was conducted in a double-wall sound-attenuating chamber
(Industrial Acoustics Co.,
Bronx, NY). A computer workstation and amplifier (Harmon-Kardon
3380, Stamford, CT)
outside of the booth interfaced with a 17-in touch-screen
monitor (Elo LCD TouchSystems,
Berwyn, PA) and two wall-mounted loudspeakers (Electro-Medical
Instrument Co., Mississauga,
ON) inside the booth. The touch-screen monitor was used for
presenting instructions for all tasks
and for recording participants’ responses. The loudspeakers were
mounted at the corners of the
sound booth, each located at 45 degrees azimuth to the
participant, and the touch-screen monitor
was placed at the midpoint. Sound files were presented between
60 and 65 dB, according to the
preferences of each participant. One CI user (CI-2) requested
sound levels up to 75 dB. CI
participants were free to alter the settings on their processor
in the course of the test session.
2.3 Test Battery
Trials for the Metric Task (from Hébert & Cuddy, 2002), the
Rhythmic subtest of the Montreal
Battery for Evaluation of Amusia (MBEA; Peretz, Champod &
Hyde, 2003), the Distorted Tunes
Test (DTT; Drayna, Manichaikul, de Lange, Snieder & Spector,
2001), the Music Emotion Task
(Veillard, Peretz, Gosselin, Khalfa, Gagnon & Bouchard,
2007), the Diagnostic Analysis of
Nonverbal Accuracy Scale 2 (DANVA2; Nowicki & Duke, 1994;
Baum & Nowicki, 1998), and
the individualized Familiar Music Task were presented via a
customized program created with
Affect 4.0 (Hermans, Clarysse, Baeyens & Spruyt, 2005;
Spruyt, Clarysse, Vansteenwegen,
Baeyens & Hermans, 2010). FLXLab 2.3 software (Haskell,
2009) was used to arrange the
presentation of the Word Recognition, Pitch-Matching, and
Interval-Matching tasks. The entire
Clinical Assessment of Music Perception (CAMP) test, which was
designed for cochlear implant
users (Kang et al., 2009), was also adminstered.
-
7
2.3.1 Metric task
The rhythms comprising this task were the strong-meter rhythms
from Hébert and Cuddy (2002).
These rhythms were created with SoundEdit 16, version 2.0. A
temporal interval was defined as
the onset-to-onset time (IOI) of successive events, with all
events consisting of the sound of a
snare drum. The basic IOI was 200 ms, and IOIs varied in a
1:2:3:4 ratio, with IOIs of 200, 400,
600, and 800 ms. Each standard rhythm consisted of a different
permutation of nine IOIs (five
IOIs of 200 ms, two of 400 ms, one of 600, and one of 800 ms).
All tones were of equal intensity
(i.e., no amplitude accents) and duration (100 ms). To create
strong metric patterns, longer IOIs
occurred on the beat. There were 4 practice trials (2 same, 2
different) with visual feedback
(correct, incorrect) provided on the monitor followed by 20 test
trials (10 same, 10 different)
presented in random order with no feedback. On each trial,
participants received a standard and
comparison drum pattern, and they were required to judge whether
they were the same or
different. On “same” trials, the standard and comparison
patterns were identical. On “different”
trials, one 400-ms IOI from the standard pattern was replaced by
an 800-ms IOI. Participants
responded by touching “same” or “different” on the
touch-sensitive monitor. They also touched
the monitor to proceed to the following trial, at their own
pace.
2.3.2 Rhythm task
The principal modification to the Rhythmic subtest of the MBEA
(Peretz et al., 2003) was the
addition of accompaniment, as described below. The test
consisted of 31 trials without feedback
preceded by training trials consisting of two examples with
feedback. Participants listened to
two tonal melodies and judged whether they were the same or
different. Differences consisted of
alterations in the duration of two adjacent tones, which changed
the rhythmic grouping but not
the meter or number of tones. Rhythmical patterns varied across
melodies. The melodies spanned
a total frequency range of 247 (B3) to 988 Hz (B5), with the
smallest range being 247 to 311 (E-
flat-4) Hz, and the largest range 247 to 784 (G5) Hz. Melodies
had 7 to 21 notes and were 3.8 to
6.4 s in duration (M = 5.1 s), depending on the tempo (100, 120,
150, 180, and 200 bpm). Tone
durations varied from 150 to 1800 ms depending on the rhythm and
tempo of each melody.
Synthesized piano versions of the melodies were used.
-
8
For the present purposes, accompaniment consisting of sampled
bass, guitar (strummed
chords), and drum kit sounds created by means of Cakewalk Music
Creator (Version 5.0.4.23;
Roland, Hamamatsu, Japan) was added to all of the melodies.
Amplitude was standardized for
each instrumental track across all melodies. Participants were
told that accompaniment had been
added to increase the difficulty of the task. They were asked to
base their judgments of similarity
or difference entirely on the piano melody. Participants called
for trials by touching the monitor
and entered their responses (same or different) on the
monitor.
2.3.3 Distorted Tunes Test
This test (Drayna et al., 2001) required participants to judge
whether synthesized piano
performances of 26 short melodies (12-26 notes) that are
well-known in the U.K. and North
America were correct (no pitch errors) or distorted (one or more
pitch errors). Of the 26 tunes, 9
were played correctly, and 17 were distorted by pitch changes
(i.e., errors) in 2-9 notes, within
one or two semitones of the correct note but maintaining the
melodic contour (rise and fall) of
the normal melody. The errors in the melodies resulted in
out-of-key notes in all but one melody
(stimulus no. 13). All melodies in the DTT were unaltered in
rhythm. The majority of tunes (17
out of 26) were played incorrectly, but there is no indication
of performance differences on intact
or distorted versions (Drayna et al., 2001).
2.3.4 Musical emotion test
This task, from Veillard et al. (2007), required participants to
identify the predominant emotion
conveyed by short musical excerpts as happy, sad, angry, or
scary. The excerpts, representing
five of the most readily identified excerpts from each emotion
category, as determined in a
preliminary study (Hunter, Schellenberg, & Stalinski,
submitted), were MIDI files set to piano
timbre. The happy excerpts were in the major mode with a mean
tempo of 137 beats per minute
(bpm) and the melodic line in a medium-to-high pitch range. The
sad excerpts were in the minor
mode, with a mean tempo of 44 bpm, medium pitch range, and
sustain pedal. The peaceful
excerpts were in the major mode, with an intermediate tempo of
69 bpm, a medium pitch range,
and also the sustain pedal. The scary excerpts had minor chords
on the third and sixth degree, a
mean tempo of 95 bpm, and a low-medium pitch range. Mean
stimulus duration was 13.3 s for
all emotional categories.
-
9
2.3.5 Diagnostic Analysis of Nonverbal Accuracy 2
The Adult Paralanguage subtest of the DANVA2 (Baum &
Nowicki, 1998) assessed the ability
to perceive emotion through non-verbal speech cues. In this
test, a semantically neutral sentence
(“I’m going out of the room now, but I’ll be back later”) was
spoken with happy, sad, angry, or
fearful intentions at two levels of emotional intensity by a
male and female actor.
2.3.6 Open-set word recognition
As a check on basic speech perception skills, CI users and NH
listeners were required to repeat
20 isolated consonant-vowel-consonant (CVC) words (see Table 2)
produced by a female
speaker. This task, like others in the battery, was
self-administered and self-paced. Each stimulus
word was preceded by a visual warning signal on the monitor (+),
and participants’ responses
were recorded.
Table 2. List of CVC words
back beach chain cup doll
fan food gum jar leg
love map meat nut pen
pig run sit sun talk
2.3.7 CAMP test
This music perception test (Kang et al., 2009) had subtests of
pitch direction discrimination,
melody recognition, and timbre recognition. The pitch subtest
used an adaptive procedure (1-up
1-down) to determine the threshold for pitch direction
discrimination within the range of 1 to 12
semitones. On each trial, listeners indicated whether the first
or second of two tones was higher
in pitch. The melody subtest assessed recognition of widely
known melodies presented without
rhythmic cues (i.e., all tones of equal duration). On each
trial, listeners identified the melody
-
10
from a set of 12 alternatives. In the timbre subtest, listeners
heard a five-note sequence (the same
one on all trials) and were required to identify the instrument
from a set of eight alternatives.
Stimuli for the pitch direction and melody subtests consisted of
synthesized, complex tones with
uniform spectral envelopes to preclude temporal envelope cues.
Stimuli for the timbre subtest
consisted of recordings of professional musicians playing real
instruments. The pitch subtest was
preceded by four practice trials and the melody and rhythm
subtests were preceded by training
sessions, in which participants were required to listen to each
stimulus twice before beginning
the test phase.
2.3.8 Familiar music task
Stimuli for this task were personalized for CI users and NH
listeners based on music that was
most familiar to them. Prior to their laboratory visit,
participants provided a list of up to 10
musical selections (title, album and recording artist) that they
heard regularly. Five selections
were included in the test, along with five unfamiliar selections
from the same genre (as listed on
ITunes) and with similar tempi. The familiar music task had
three conditions: (1) no rhythm, (2)
melody only, and (3) original instrumental. The original
instrumental versions consisted of 10-s
excerpts with salient melodic content from each selection, which
were extracted with Audacity
software (Version 1.3.11 Beta). If the musical selection did not
have 10 s without vocal content,
the vocals were removed with Vocal Remover Version 2 plugin from
Audacity. Melodic content
from all selections was transcribed to produce two monophonic
WAV files per selection – a
melody version and a no-rhythm version. These excerpts were
produced with a synthesized flute
timbre from Cakewalk Music Creator. In contrast to the melody
version, which maintained the
rhythm, the no-rhythm version was isochronous (i.e., all tones
of equal duration). The original
pitch durations were maintained in the no-rhythm version by
means of repeated tones at the
pitches in question. On each trial, participants listened to the
selection and identified it from a set
of six alternatives, which consisted of the five familiar
musical pieces and “none of the above.”
The conditions were administered in fixed order from most to
least difficult: (1) no rhythm; (2)
melody; and (3) original instrumental.
-
11
2.3.9 Pitch- and interval matching
The stimuli for this task consisted of eight pitches 1-2 s in
duration sung by a man and woman
and eight ascending intervals sung by the same individuals in a
legato (continuous or
uninterrupted) manner. The male stimuli ranged from B3 (246.942
Hz) to B4 (493.883 Hz), and
the female stimuli ranged from B4 to B5 (987.767 Hz). Each pitch
and interval stimulus was
presented twice in a predetermined order, with the
pitch-matching task presented first.
Participants were required to sing back what they heard, and
their responses were recorded by
means of FLXLab. The intervals always began on the first degree
of the scale (B3 for male
stimuli and B4 for female stimuli). Only pitches from the key of
B major were used, which
resulted in the following intervals: unison, octave, major 2nd,
3rd, 6th and 7th, and perfect 4th
and 5th. Pitches and intervals of the imitations were calculated
by means of Praat software
(Version 5.1.43; Boersma & Weenink, 2010).
2.4 Procedure
Prior to their laboratory visit, implant users completed a
questionnaire (see Appendix A) that
included information about demographic background (e.g., history
of hearing loss, implant
experience, education, languages spoken), musical background
(e.g., musical training, music
listening habits before and after their hearing loss, music
enjoyment) and familiar musical
selections. NH participants completed a questionnaire about
their musical background and
subjective experience of music (see Appendices B and C) just
before the test session.
Test sessions with CI users began with a semi-structured
interview designed to elicit
information about their subjective experience of music (see
Appendix D). All interviews were
recorded with a Sony Net MD Walkman (MZ-N707 model) and a Sony
electret condenser
microphone (ECM-DS70P model). Once the interview was completed,
participants were
escorted to the sound-attenuating booth for administration of
the test battery. The experimenter
provided instructions before each component of the battery.
These instructions were repeated on
the touch-screen monitor prior to each task. Participants were
also told that the sounds could be
made louder or softer, according to their preference. Tasks were
presented in fixed order.
Participants were told that the pitch- and interval-matching
tasks, which were the in the test
battery, were strictly optional.
-
12
3 Results and Discussion
Due to the small sample size and large individual differences
among CI users, we examined
performance individually for each task, noting the CI users who
performed within one SD of the
mean for NH listeners, those who performed within two SDs, and
so on. On the basis of previous
research, CI users were expected to perform much better on tests
of rhythm and meter and on
other tasks based on timing cues than on those based on pitch
cues (Cooper et al., 2008; Kang et
al., 2009; Kong et al., 2004).
3.1 Open-Set Word Recognition
As one would expect, performance of the NH group was at ceiling
(see Figure 1) such that there
was no variance in the data. CI users also performed well on
this task, with scores ranging from
100% correct (CI-2) to a low of 20% correct (CI-4). Performance
on these isolated monosyllabic
words sheds light on how well each CI user was using bottom-up
cues that are relevant to speech
perception. It should be emphasized, however, that the CI
participants were uniformly excellent
at perceiving speech in quiet backgrounds when contextual cues
were available, as confirmed in
lengthy, individual interviews. Variability on open-set
recognition tasks has been reported in a
number of other studies (Loizou, Poroy, & Dormann, 2000;
Vandali, Whitford, Plant, & Clark,
2000). CI-4, who had the poorest performance on this task, had
the longest delay from the time
her hearing aids became ineffectual (i.e., no usable residual
hearing) until implant surgery. The
top performer, CI-2, had a number of advantages, including
professional knowledge of hearing
and assistive technologies as well as residual low-frequency
hearing (at 250 and 500 Hz) in his
unimplanted ear, which was selectively amplified. Zhang, Dorman,
and Spahr (2010) have
documented the contribution of low-frequency acoustic hearing to
the recognition of
monosyllabic words.
3.2 CAMP
Pitch-detection thresholds are illustrated in Figure 2, whereas
melody recognition and timbre
recognition are illustrated in Figures 3 and 4, respectively.
The mean threshold for pitch-
direction identification for NH listeners was 1.3 semitones (SD
= 0.8), whereas their average on
the melody-recognition task was 88.0% (SD = 10.0%) and their
average result on the timbre-
recognition task was 85.6% (SD = 16.0%). The means for the CI
users group were 4.6 semitones
-
13
Figure 1. Mean score and standard deviation on the Speech
Perception Task for normally hearing (NH) listeners and individual
scores for cochlear implant (CI) users.
Figure 2. Mean score and standard deviation on the CAMP Pitch
Task for normally hearing (NH) listeners and individual scores for
cochlear implant (CI) users.
-
14
Figure 3. Mean score and standard deviation on the CAMP Melody
Recognition Task for normally hearing (NH) listeners and individual
scores for cochlear implant (CI) users. The dotted line represents
chance performance.
Figure 4. Mean score and standard deviation on the CAMP Timbre
Recognition Task for normally hearing (NH) listeners and individual
scores for cochlear implant (CI) users. The dotted line represents
chance performance.
-
15
(SD = 2.7) for the pitch-ranking task, 25.7% correct for the
melody task (SD = 25.5), and 45.8%
for the timbre task (SD = 19.5). Results for both groups were
very similar to those reported
previously by the developers of the test (Kang et al., 2009).
Because the CAMP tests examine
the ability to perceive pitch and timbre cues, it is not
surprising that most of the CI users did not
do well on these tasks.
Two CI users (CI-1 and CI-2) managed to perform particularly
well on pitch-direction
identification, falling within one SD of the mean of the NH
group. The Melody task, which
excluded all timing cues, proved to be more difficult. In fact,
two CI users (CI-5 and CI-6) opted
to discontinue the task because of its extreme difficulty.
Moreover, no CI user was able to obtain
a score within two SDs of the NH mean, although CI-2’s
performance was substantially better
than that of the other CI users. His score was 63.9%, whereas
the average of the scores of the
three other CI users was 13.0%. CI-2 also scored much higher
than other CI users on the timbre
identification task, obtaining a score of 83.3% correct, which
was near the NH mean. None of the
other CI users had a score within two SDs of the NH mean,
although CI-3 came close. The
amplified residual hearing of CI-2 undoubtedly accounts for his
success and for his ability to
play in a musical ensemble. The contribution of hearing aids in
the unimplanted ear to music
perception has been noted previously (Looi, McDermott, McKay,
& Hickson, 2007; Turner,
Reiss & Gantz, 2008).
3.3 Distorted Tunes Test
The DTT comprised 26 questions and two response options on each
trial, such that chance
performance was a score of 13. As in the original study that
used the DTT (Drayna et al., 2001)
and an additional study by the same research team (Jones et al.,
2009), the scores of NH listeners
were near ceiling (M = 24.7, SD = 1.3; see Figure 5). Because
the DTT comprises traditional
North-American folk melodies, our CI users, who were on average
much older than our control
group (mean age of 62.7 vs 29.0), would have been more familiar
with these melodies before
they became deaf. Nonetheless, CI users had extreme difficulty
on this task. Their mean score
was 11.8 correct (SD = 2.8), and scores for all of the CI users
were more than two SDs below the
NH mean and near chance levels. In fact, the highest score was
16 correct (CI-2). Because
mistunings on the DTT (except for one) are created by using
pitches outside the key of each
melody, the findings indicate that CI users are unable to use
tonality-related cues when
-
16
perceiving music. This interpretation is consistent with
previous performance of CI users on the
Scale subtest of the MBEA, which was at chance levels (Cooper et
al., 2008).
It is notable that CI-2, despite his good pitch resolution and
reasonable performance on
other tasks, was unable to do this task, which involved
comparing current melodies with long-
term representations of those melodies or making judgments based
on tonality. As noted, the
pitch errors in this test are relatively small (one or two
semitones). Considering the mean score
of CI users on the CAMP pitch-ranking task (threshold of 4.6
semitones), it is not surprising that
CI users were unable to perceive mistunings in the DTT melodies.
The authors of the DTT
created these errors to be salient by virtue of their violations
of tonality. Thus, it is not surprising
that these violations are not salient to CI users.
Figure 5. Mean score and standard deviation on the Distorted
Tunes Test for normally hearing (NH) listeners and individual
scores for cochlear implant (CI) users. The dotted line represents
chance performance.
-
17
3.4 Familiar Music Task
Scores for this task were the number of correct answers out of
10, converted into percent correct
scores. It was possible to generate individualized materials for
only five participants from the
NH group. Mean scores were 74.0% (SD = 15.2%) in the No-rhythm
condition (Figure 6),
92.0% (SD = 4.5%) in the Melody (with timing cues) condition
(Figure 7), and 94.0% (SD =
5.5%) in the Instrumental condition (Figure 8), which featured
all or most cues from the original
recordings, except for the lyrics for selections involving
songs. CI users’ scores were
exceedingly low. Moreover, they were lowest in the No-Rhythm
condition (M = 26.7%, SD =
25.2%), slightly higher in the Melody condition (M = 35.0%, SD =
19.1%), and highest in the
Instrumental condition (M = 70.0%, SD = 18.3%). Two CI users
were excluded from
consideration because they provided artists with whom they were
familiar (e.g., Louis
Armstrong, Frank Sinatra) but no specific musical selections. Of
the four remaining CI users,
one discontinued the No-Rhythm condition because of its
difficulty. CI-1 and CI-2 scored below
two SDs of the NH mean for this condition. Although CI-4 managed
to score within two SDs of
Figure 6. Mean score and standard deviation on the No-Rhythm
Condition of the Familiar Music Task for normally hearing (NH)
listeners and individual scores for cochlear implant (CI)
users.
-
18
the NH mean, she did so only by responding “none of the above”
for all of the trials. Obviously,
she was unable to recognize any melodies without rhythmic cues.
Although CI users fared better
in the Melody condition than in the No-Rhythm condition, all
four failed to score within two SDs
of the mean for NH listeners. In the Instrumental condition,
CI-2 obtained a score similar to the
NH mean (90.0% versus 94.0%, respectively). Although CI-4 and
CI-1 obtained higher scores in
the Instrumental condition than in the other two conditions,
they were still more than two SDs
below the NH mean.
Figure 7. Mean score and standard deviation on the Melody
Condition of the Familiar Music Task for normally hearing (NH)
listeners and individual scores for cochlear implant (CI)
users.
-
19
The Familiar Music Task was created specifically for this study.
The expectation was that
the use of highly familiar music would generate better
performance than one would predict based
on the available literature. In fact, CI children have shown
some success in the recognition of
specific recordings that they hear regularly (Vongpaisal, Trehub
& Schellenberg, 2006 & 2009)
even though such children are generally unsuccessful at
recognizing generic versions of
culturally familiar tunes (Olszweski, Gfeller, Froman, Stordahl,
& Tomblin, 2005; Stordahl,
2002). However, this was not the case for the current group of
adult CI users. Of the six CI users
in the present study, three (CI-1, CI-5 and CI-6) reported in
their interview that the lyrics were
the most salient part of their music listening experiences.
However, lyrics were excluded from
the test materials, even when recordings with vocals were
selected as familiar music, because
they provided obvious cues to the identity of the music.
CI-4, who listened to classical music and attended concerts
frequently, was unable to
recognize the original recordings (same music and performers)
that she heard regularly. CI-6 was
also unable to recognize the four instrumental pieces that were
among her 10 selections, which
suggests that her at-home listening experiences are guided and
enriched by knowledge of what
Figure 8. Mean score and standard deviation on the Instrumental
Condition of the Familiar Music Task for normally hearing (NH)
listeners and individual scores for cochlear implant (CI)
users.
-
20
she is playing. CI-2, the “star” performer in the present study,
indicated that he listens especially
closely to the bass line in music. This follows, perhaps, from
programming his hearing aid to
capitalize on his residual low-frequency hearing. CI-2 is also a
bass player who performs with an
amateur blues/rock group. It is nevertheless impressive that
this participant was as proficient as
NH listeners at identifying the familiar instrumental
excerpts.
During her interview, participant CI-6 shed light on factors
contributing to her musical
preferences. She stated that, in order to enjoy music, it had to
have meaning, such as a narrative.
For example, she very much enjoyed the lyrics in a number of the
selections she submitted.
Although she also selected instrumental pieces, some of them
were orchestral works with
underlying narratives. For example, Symphony No. 11 by Dmitri
Shostakovich, entitled “In The
Year 1905,” depicts the Russian revolution. Another of her
selections, the orchestral work
“Finlandia” by Jean Sibelius, depicts the Finnish struggle to
break free from the Russian empire.
Because CI users do not have access to the acoustic details
available to NH listeners, they may
find other ways of enjoying music. The enjoyment of CI-6 was
enriched by a narrative linked to
the overall structure of the musical work rather than its
melodies or harmonies. CI-6 described
hearing the Cossacks charging on their horses in the work by
Shostakovich, and the struggles and
the triumph of the Finnish people in the Sibelius piece.
Another factor that may have contributed to CI users’ difficulty
at identifying the
material was the 10-second duration of the excerpts, which posed
no problem for NH listeners. It
is possible that they would be somewhat more successful with
longer samples of the music.
3.5 Metric Task & Modified MBEA Rhythm Task
Because the Metric task comprised 20 questions and each trial
had two response options,
chance responding was a score of 10. The mean of the NH group
was 17.1 (SD = 3.4; see Figure
9), which is similar to the mean of a NH group tested on the
same task in a previous study
(Hopyan, Schellenberg & Dennis, 2009). CI-1 received a
perfect score on this task. CI-2 and CI-
3 scored within one SD below the NH mean, and CI-4 and CI-5
scored within two SDs. CI-6 was
below two SDs and also below chance levels for this task. In
short, the majority of CI users (5 of
6) were within two SDs of the mean for NH listeners, which is in
line with CI users’ previous
success in discriminating simple rhythmic patterns (Gfeller et
al., 1997; Kong et al., 2004).
-
21
The modified subtest of the MBEA had 31 trials and two response
options on each trial,
such that chance performance was a score of 15.5. The NH group
mean was 26.9 (SD = 3.6, see
Figure 10), which is virtually identical to a sample of
individuals with normal music perception
skills reported by Peretz et al. (2003) who were tested on a
similar task without accompaniment
(M = 27.0, SD = 2.1). Similar performance across studies
indicates that the additional
instrumentation created for the purpose of this study did not
impair the performance of NH
listeners. By contrast, the average performance of CI users on
our modified task was only 17.0
(SD = 2.1), which is substantially lower than the mean obtained
by CI users tested by Cooper et
al. (2008; approximately 24 correct) on the original MBEA rhythm
task. Although CI-2 was
slightly less than two SDs below the NH mean, the other CI users
were near or below chance
levels, which confirms that the additional instrumentation
impeded their ability to perceive the
rhythm of the melodic line.
Figure 9. Mean score and standard deviation on the Metric Task
for normally hearing (NH) listeners and individual scores for
cochlear implant (CI) users. The dotted line represents chance
performance.
-
22
Although CI users fared as well as NH listeners on the original
version of this rhythm
discrimination task, which involved monophonic piano melodies
(Cooper et al., 2008), their
rhythm discrimination was impaired when there were multiple
streams of auditory information.
In fact, almost all of the CI users performed near chance level.
This finding suggests that CI
users would have difficulty discerning the rhythms encountered
in their everyday experience of
music.
3.6 Music Emotion & DANVA2
The Music Emotion task comprised 20 questions and 4 response
options on each trial, such that a
score of five correct responses corresponded to chance
responding. Once again, NH listeners
were near ceiling on this task (M = 19.2, SD = 1.4; see Figure
11), which is slightly higher than
the results reported by Hunter et al. (submitted) for adult
listeners, who had an average of 16.7
correct. All CI users were more than two SDs below the NH mean,
with a mean of 12.7 (SD =
3.6).
Figure 10. Mean score and standard deviation on the Modified
MBEA Rhythm Task for normally hearing (NH) listeners and individual
scores for cochlear implant (CI) users. The dotted line represents
chance performance.
-
23
Because CI users are better able to perceive timing cues than
pitch cues, we examined the
possibility that CI users could interpret arousal, which is
based largely on tempo cues, better than
valence, which is based on mode (major/minor) and
consonance/dissonance cues. Thus, we
combined the response options based on arousal: happy or scary
vs. sad or peaceful (see Table 3
for arousal scores). For three of the CI users (CI-2, CI-4 and
CI-5), a majority of the errors (over
50%) on this task involved confusions between stimuli that
contrasted in valence but were
similar in arousal. These findings suggest that tempo cues play
a substantially greater role than
mode cues in CI users’ perception of emotion in music. This
interpretation is consistent with
reports of adequate tempo perception in CI users (Kong et al.,
2004). Tempo cues are also more
important than mode cues for young children (Dalla Bella,
Peretz, Rousseau, & Gosselin, 2001),
not because of pitch resolution difficulties but because they
have not yet learned Western
musical conventions about mode.
The DANVA2 comprised 24 trials and four response options on each
trial (happy, angry,
sad, and fearful) such that a score of six correct corresponded
to chance responding. The mean
for the NH listeners was 19.3 (SD = 2.3; see Figure 12), which
is similar to the mean reported by
Nowicki (2006; M = 18.0, SD = 2.9). The average score for the CI
users was only 10.8 (SD =
3.3). Only CI-2 and CI-6 performed within two SDs of the NH
mean, with the remaining CI
users having lower scores and three performing at close to
chance levels (CI-3, CI-4, CI-5).
Performance on the DANVA2 by child CI implant users in the study
by Hopyan-Misakyan et al.
(2009) was similar to adult CI users in the present study in
that both groups were unsuccessful in
differentiating the vocal emotions.
The DANVA2, which has been used widely (Nowicki, 2006), is
intended to be a
challenging test, with average NH scores ranging from 14 to 18.5
out of 24. Among its
advantages is that it allows specially gifted individuals to
achieve higher scores than the
population mean. However, this test may not be the most
appropriate means of assessing CI
users’ access to emotion cues in speech. A test involving a
greater range of emotional
expressiveness would enable us to learn more about this skill in
CI users.
-
24
Table 3. Music Emotion Arousal Scores Participant Test score
(20) Modified arousal score (20) Valence errors
CI-1 17 18 33% CI-2 13 18 71% CI-3 15 17 40% CI-4 14 18 67% CI-5
7 12 38% CI-6 10 16 60%
Figure 11. Mean score and standard deviation on the Music
Emotion Task for normally hearing (NH) listeners and individual
scores for cochlear implant (CI) users. The dotted line represents
chance performance.
-
25
3.7 Pitch- and Interval-Matching Task
Only CI users were asked to complete the pitch- and
interval-matching tasks, which were
described as strictly optional. With the exception of one CI
user, who was short of time, all
agreed to complete the matching tasks. The overwhelming majority
of NH individuals can match
pitches within one semitone (Moore, Estis, Gordon-Hickey, &
Watts, 2008). For CI users, the
mean error in pitch matching (Figure 13) was 3.9 semitones (SD =
3.1). Only CI-2 performed
within the expected range of NH listeners, with a mean pitch
error of 1.1 semitones. Performance
in the interval-matching task (Figure 14) was similar. Errors on
interval matching (Figure 14)
were comparable to those on pitch matching (M = 3.1 semitones,
SD = 2.0). Again, CI-2
performed surprisingly well, with a mean error of 1.0 semitones
on interval matching, which is in
line with his low pitch-ranking threshold on the CAMP test (1.6
semitones).
Figure 12. Mean score and standard deviation on the DANVA2 Adult
Vocal Emotion Task for normally hearing (NH) listeners and
individual scores for cochlear implant (CI) users. The dotted line
represents chance performance.
-
26
Figure 13. Individual average pitch deviations in semitones on
the Pitch-Matching Task for cochlear implant (CI) users.
Figure 14. Individual average pitch deviations in semitones on
the Interval-Matching Task for cochlear implant (CI) users.
-
27
3.8 Conclusion
In sum, postlingually deafened adult CI users performed well
below the level of the NH control
group on most tasks in the present study. Their performance was
especially poor on tasks that
relied strongly on pitch cues, such as the DTT, isochronous
melody tasks, familiar melody task,
pitch ranking, and pitch matching. They had more success on the
simple rhythm discrimination
task but not on the more complex rhythm discrimination task.
They also had poor results on the
emotion discrimination tasks, which required the joint use of
pitch and timing cues.
As in most studies of CI users, there were large individual
differences in performance.
CI-2 performed considerably better than other CI users,
especially on the pitch-ranking and
pitch-matching tasks. Although his musical background may have
played some role, it is likely
that amplified residual hearing in his unimplanted ear made the
most important contributions to
his success on the tasks involving pitch. Along with musical
training and residual hearing, CI-2
had the further advantage of formal training in audiology and
familiarity with hearing aid
technology. As he put it, he programmed his own hearing aid to
“act like a subwoofer,” which
enables him to maximize his perception of music and speech. In
his interview, CI-2 indicated
that neither his implant nor his hearing aid alone provided a
satisfactory representation of sound
but together they provided a credible and highly enjoyable
rendition of music. In short, the whole
was a lot better than the sum of its parts. Plans for re-testing
CI-2 with his implant alone will
provide a clearer picture of the independent contributions of
implant and hearing aid.
CI-4 had extensive musical training (piano) and even considered
a career as a musician
when she was a young woman with normal hearing. Her progressive
hearing loss over the years
and a long period of very poor auditory reception with hearing
aids seemed to erase any potential
benefit from her training and knowledge of music. For CI-2, by
contrast, gradual hearing loss
began at about 5 years of age and his hearing aids functioned
effectively for music listening until
approximately five years before receiving his implant.
Plans to enlarge the sample will make it possible to identify
links between various
background variables and performance on music processing tasks
such as these. It would be of
interest to determine whether limited training enhances music
processing in CI users and their
-
28
ability to derive pleasure from music. Such training may also
have favorable consequences for
other auditory but non-musical tasks. These questions can be
addressed in future research.
-
29
References
Baum, K., & Nowicki, S., Jr. (1998). Perception of emotion:
Measuring decoding accuracy of
adult prosodic cues varying in intensity. Journal of Nonverbal
Behavior, 22, 89 – 109.
Boersma, P., & Weenink, D. (2010). Praat: Doing phonetics by
computer [computer software].
Retrieved August 17, 2010, from
http://www.fon.hum.uva.nl/praat/download_win.html.
University of Amsterdam, The Netherlands.
Cooper, W.B., Tobey, E., & Loizou, P.C. (2008). Music
perception by cochlear implant and
normal hearing listeners as measured by the Montreal Battery for
Evaluation of Amusia.
Ear & Hearing, 29, 618 – 626.
Dalla Bella, S., Giguère, J.-F., & Peretz, I. (2009).
Singing in congenital amusia. Journal of the
Acoustical Society of America, 126, 414 – 424.
Dalla Bella, S., Peretz, I., Rousseau, L. & Gosselin, N.
(2001). A developmental study of the
affective value of tempo and mode in music. Cognition, 80, B1 –
B10.
Drayna, D., Manichaikul, A., de Lange, M., Snieder, H., &
Spector, T. (2001). Genetic
correlates of musical pitch recognition in humans. Science, 291,
1969 – 1972.
Fetterman, B.L., & Domico, E.H. (2002). Speech recognition
in background noise of cochlear
implant patients. Otolaryngology – Head and Neck Surgery, 126,
257 – 263.
Firszt, J.B., Holden, L.K., Skinner, M.W., Tobey, E.A.,
Peterson, A., Gaggl, W., et al. (2004).
Recognition of speech presented at soft to loud levels by adult
cochlear implant recipients
of three cochlear implant systems. Ear & Hearing, 25, 375 –
387.
Foxton, J.M., Nandy, R.K., & Griffiths, T.D. (2006). Rhythm
deficits in ‘tone deafness.’ Brain
and Cognition, 62, 24 – 29.
Fu, Q.-J., Nogaki, G., & Galvin, J.J., III (2005). Auditory
training with spectrally shifted speech:
Implications for cochlear implant patient auditory
rehabilitation. Journal for the
Association of Research in Otolaryngology, 6, 180 – 189.
-
30
Galvin, J.J., III, Fu, Q.-J., & Nogaki, G. (2007). Melodic
contour identification by cochlear
implant listeners. Ear & Hearing, 28, 302 – 319.
Galvin, J.J., III, Fu, Q.-J., & Shannon, R.V. (2009).
Melodic contour identification and music
perception by cochlear implant users. Annals of the New York
Academy of Sciences,
1169, 518 – 533.
Gfeller, K., Christ, A., Knutson, J.F., Witt, S., Murray, K.T.,
& Tyler, R.S. (2000). Musical
backgrounds, listening habits and aesthetic enjoyment of adult
cochlear implant
recipients. Journal of the American Academy of Audiology. 11,
390 – 406.
Gfeller, K., Christ, A., Knutson, J.F., Woodworth, G., Witt, S.
& DeBus, B. (1998). Timbral
recognition and appraisal by adult cochlear implant users and
normal-hearing adults.
Journal of the American Academy of Audiology, 9, 1 – 19.
Gfeller, K., Turner, C., Oleson, J., Zhang, X., Gantz, B.,
Froman, R., et al. (2007). Accuracy of
cochlear implant recipients on pitch perception, melody
recognition, and speech reception
in noise. Ear & Hearing, 28, 412 – 423.
Gfeller, K., Witt, S., Stordahl, J., Merh, M., & Woodworth,
G. (2000). The effects of training on
melody recognition and appraisal by adult cochlear implant
recipients. Journal of the
Academy of Rehabilitative Audiology, 33, 115 – 138.
Gfeller, K., Woodworth, G., Robin, D. A., Witt, S., Knutson,
J.F. (1997). Perception of rhythmic
and sequential pitch patterns by normally hearing adults and
adult cochlear implant
users. Ear & Hearing, 18, 252 – 260.
Hartmann, R., & Kral, A. (2004). Central Responses to
Electrical Stimulation. In F.-G. Zeng,
A.N. Popper, & R.R. Fay (Eds.), Cochlear Implants: Auditory
Prostheses and Electric
Hearing (pp. 213 – 285). New York: Springer.
Haskell, T.R. (2009). FLXLab 2.3 [computer software and manual].
Retrieved February 28,
2010, from http://xlab.sourceforge.net
-
31
Hébert, S., & Cuddy, L.L. (2002). Detection of metric
structure in auditory figural patterns.
Perception & Psychophysics, 64, 909 – 918.
Hermans, D., Clarysse, J., Baeyens, F., & Spruyt, A. (2005).
Affect Version 4.0 [Computer
software]. Retrieved February 4, 2010, from
http://www.psy.kuleuven.ac.be/leerpsy/
affect4. University of Leuven, Belgium.
Hopyan, T., Schellenberg, E.G., & Dennis, M. (2009).
Perception of strong-meter and weak-
meter rhythms in children with spina bifida meningomyelocele.
Journal of the
International Neuropsychological Society, 15, 521 – 528.
Hopyan-Misakyan, T. M., Gordon, K. A., Dennis, M., & Papsin,
B.C. (2009). Recognition
of affective speech prosody and facial affect in deaf children
with unilateral right
cochlear implants. Child Neuropsychology, 15, 136 – 146.
Hunter, P.G., Schellenberg, E.G., & Stalinski, S.M.
(submitted). Emotions expressed by music:
Age and gender difference in identification accuracy and
preferences.
Jones, J.L., Lucker, J., Zalewski, C., Brewer, C., & Drayna,
D. (2009). Phonological processing
in adults with deficits in musical pitch recognition. Journal of
Communication Disorders,
42, 226 – 234.
Juslin, P.N., & Laukka, P. (2003). Communication of emotions
in vocal expression and music
performance: Different channels, same code? Psychological
Bulletin, 129, 770 –814.
Kang, R., Nimmons, G.L., Drennan, W., Lognion, J., Ruffin, C.,
Nie, K., et al. (2009).
Development and validation of the University of Washington
Clinical Assessment
of Music Perception test. Ear & Hearing, 30, 411–418.
Kong,Y.-Y., Cruz, R., Ackland-Jones, J., & Zeng, F.-G.
(2004). Music perception with temporal
cues in acoustic and electric hearing. Ear & Hearing, 25,
173 – 185.
McDermott, H.J., & Looi, V. (2004). Perception of complex
signals, including musical sounds,
with cochlear implants. International Congress Series, 1273,
201–204.
-
32
Lassaletta, L., Castro, A., Bastarrica, M., Pe´Rez-Mora, R.,
Madero, R., De Sarria, J., et al.
(2007). Does music perception have an impact on quality of life
following cochlear
implantation? Acta Oto-Laryngologica, 127, 682 – 686.
Laukka, P. (2006). Uses of music and psychological well-being
among the elderly. Journal of
Happiness Studies, 8, 215 – 241.
Leake, P.A., & Rebscher, S.J. (2004). Anatomical
Considerations and Long-Term Effects of
Electrical Stimulation. In F.-G. Zeng, A.N. Popper, & R.R.
Fay (Eds.), Cochlear
Implants: Auditory Prostheses and Electric Hearing (pp. 101 –
148). New York:
Springer.
Leal, M.C., Shin, Y.J., Laborde, M.-L., Calmels, M.-N., Verges,
S., Lugardon, S., et al. (2003).
Music Perception in Adult Cochlear Implant Recipients. Acta Oto-
Laryngologica, 123,
826 – 835.
Loizou, P.C., Poroy, O., & Dormann, M. (2000). The effect of
parametric variations of cochlear
implant processors on speech understanding. Journal of the
Acoustical Society of
America, 108, 790 – 802.
Looi, V., McDermott, H., McKay, C., & Hickson, L. (2004).
Pitch discrimination and melody
recognition by cochlear implant users. International Congress
Series, 1273, 197 – 200.
Looi, V., McDermott, H., McKay, C., & Hickson, L. (2007).
Comparisons of quality ratings for
music by cochlear implant and hearing aid users. Ear &
Hearing, 28, 59S – 61S.
Looi, V., McDermott, H., McKay, C., & Hickson, L. (2008).
The effect of cochlear implantation
on music perception by adults with usable pre-operative acoustic
hearing. International
Journal of Audiology, 47, 257 – 268.
Looi, V., & She, J. (2010). Music perception of cochlear
implant users: A questionnaire, and its
implications for a music training program. International Journal
of Audiology, 49, 116–
128.
-
33
Meister, H., Landwehr, M., Pyschny, V., Walger, M., & Von
Wedel, H. (2009). The perception
of prosody and speaker gender in normal-hearing listeners and
cochlear implant
recipients. International Journal of Audiology, 48, 38 – 48.
Moore, R.E., Estis, J., Gordon-Hickey, S., & Watts,C.
(2008). Pitch discrimination and pitch
matching abilities with vocal and nonvocal Stimuli. Journal of
Voice, 22, 399 – 407.
Nakata, T., Trehub, S.E., Mitani, C. and Kanda, Y. (2006). Pitch
and timing in the songs of deaf
children with cochlear implants. Music Perception, 24, 147–
154.
Nowicki, S., Jr. (2006). Manual for the receptive tests of the
Diagnostic Analysis of Nonverbal
Accuracy 2 (DANVA2). Retrieved on March 28, 2010 from
http://psychology.emory.edu/
clinical/interpersonal/DANVAmanual03.doc
Nowicki, S., Jr., & Duke, M. P. (1994). Individual
differences in the nonverbal communication
of affect: The Diagnostic Analysis of Nonverbal Accuracy Scale.
Journal of Nonverbal
Behavior, 18, 9 – 35.
Olszweski, C., Gfeller, K., Froman, R., Stordahl, J., &
Tomblin, B. (2005). Familiar melody
recognition by children and adults using cochlear implants and
normal hearing children.
Cochlear Implants International, 6, 123 – 140.
Peretz, I., Champod, A.S., & Hyde, K. (2003). Varieties of
Musical Disorders: The Montreal
Battery of Evaluation of Amusia. New York Academy of Sciences,
999, 58 – 75.
Pisoni, D.B., & Cleary, M. (2004). Learning, Memory, and
Cognitive Processes in Deaf Children
Following Cochlear Implantation. In F.-G. Zeng, A.N. Popper,
& R.R. Fay (Eds.),
Cochlear Implants: Auditory Prostheses and Electric Hearing (pp.
334 – 376). New
York: Springer.
Shannon, R.V., Zeng, F.-G., Wygonski, J., Kamath, V., &
Ekelid, M. (1995). Speech recognition
with primarily temporal cues. Science, 270, 303 – 304.
-
34
Spruyt, A., Clarysse, J., Vansteenwegen, D., Baeyens, F., &
Hermans, D. (2010). Affect 4.0: A
free software package for implementing psychological and
psychophysiological
experiments. Experimental Psychology, 57, 36 – 45.
Stordahl, J. (2002). Song recognition and appraisal: A
comparison of children who use cochlear
implants and normally hearing children. Journal of Music
Therapy, 39, 2 – 19.
Turner, C.W., Reiss, L. & Gantz, B.J. (2008). Combined
acoustic and electric hearing:
Preserving residual acoustic hearing. Hearing Research, 242, 164
– 171.
Van Dijk, J.E., Van Olphen, A.F., Langereis, M.C., Mens, L.H.,
Brokx, J.P., & Smoorenburg,
G.F. (1999). Predictors of cochlear implant performance.
Audiology, 38, 109 – 116.
Vandali, A. E., Whitford, L. A., Plant, K. L., and Clark, G. M.
(2000). Speech perception as a
function of electrical stimulation rate: Using the Nucleus 24
cochlear implant system. Ear
& Hearing, 21, 608 – 624.
Veekmans, K., Ressel, L., Mueller, J., Vischer, M., &
Brockmeier, S.J. (2009). Comparison of
music perception in bilateral and unilateral cochlear implant
users and normal-hearing
subjects. Audiololgy and Neurotology, 14, 315 – 326.
Veillard, S., Peretz, I., Gosselin, N., Khalfa, S., Gagnon, L.,
& Bouchard, B. (2007). Happy, sad,
scary and peaceful musical excerpts for research on emotions.
Cognition and Emotion,
22, 720 – 752.
Vongpaisal, T., Trehub, S.E., & Schellenberg, E.G. (2006).
Song recognition by children and
adolescents with cochlear implants. Journal of Speech, Language,
and Hearing Research,
49, 1091–1103.
Vongpaisal, T., Trehub, S.E., & Schellenberg, E.G. (2009).
Identification of TV tunes by
children with cochlear implants. Music Perception, 27,
17–24.
Wilson, B. S. (2000). Cochlear Implant Technology. In J.K.
Niparko, K.I. Kirk, N.K. Mellon,
A.M. Robbins, D.L. Tucci, & B.S. Wilson (Eds.), Cochlear
implants: Principles &
Practices (pp. 109 – 118). New York: Lippincott, Williams &
Wilkins.
-
35
Wilson, B.S. (2004). Engineering Design of Cochlear Implants. In
F.-G. Zeng, A.N. Popper, &
R.R. Fay (Eds.), Cochlear Implants: Auditory Prostheses and
Electric Hearing (pp. 14 –
52). New York: Springer.
Xu, L., Zhou, N., Chen, X., Li, Y., Schultz, H.M., Zhao, X., et
al. (2009). Vocal singing by
prelingually-deafened children with cochlear implants. Hearing
Research, 255, 129 –134.
Zhang, T., Dorman, M.F, & Spahr, A.J. (2010). Information
from the voice fundamental
frequency (F0) region accounts for the majority of the benefit
when acoustic stimulation
is added to electric stimulation. Ear & Hearing, 31, 63 –
69.
-
36
Appendix
Appendix A
MUSIC & COCHLEAR IMPLANTS QUESTIONNAIRE NAME: DATE: GENDER
(M/F): AGE: E-MAIL: TEL: COUNTRY OF BIRTH: FIRST LANGUAGE: HIGHEST
EDUCATION ACHIEVED: CAUSE OF HEARING LOSS: AGE AT DIAGNOSIS: DID
YOU USE HEARING AIDS PRIOR TO IMPLANT SURGERY? IF SO, FOR HOW LONG?
(YEARS): PROGRESSIVE LOSS? (YES/NO): TYPE OF COCHLEAR IMPLANT:
PROCESSING STRATEGY: AGE AT SURGERY: ONE IMPLANT OR TWO: IF ONE,
HEARING AID IN OPPOSITE EAR? (YES/NO):
-
37
MUSIC & COCHLEAR IMPLANTS
A. HOURS OF MUSIC LISTENING PER WEEK PRIOR TO HEARING LOSS: (1)
0 hours; (2) 1 – 4 hours; (3) 4 – 7 hours; (4) 7 – 10 hours; (5) 10
hours or more. ANSWER: B. HOURS OF MUSIC LISTENING PER WEEK AFTER
HEARING LOSS BUT PRIOR TO IMPLANT SURGERY: (1) 0 hours; (2) 1 – 4
hours; (3) 4 – 7 hours; (4) 7 – 10 hours; (5) 10 hours or more.
ANSWER: C. HOURS OF MUSIC LISTENING PER WEEK SINCE IMPLANT SURGERY:
(1) 0 hours; (2) 1 – 4 hours; (3) 4 – 7 hours; (4) 7 – 10 hours;
(5) 10 hours or more. ANSWER: D1. HAVE YOU EVER PLAYED A MUSICAL
INSTRUMENT AND / OR SUNG REGULARLY? (YES/NO): D2. IF YES, WHICH
ONE(S), AND FOR HOW MANY YEARS DID YOU PLAY AND / OR SING
REGULARLY? (e.g., piano, 3 years; voice, 5 years)
Instrument Years
D3. IF YOU ARE NO LONGER PLAYING AN INSTRUMENT AND/OR SINGING,
HOW LONG HAS IT BEEN SINCE YOU LAST PLAYED AN INSTRUMENT AND/OR
SANG REGULARLY? (YEARS):
-
38
E1. HAVE YOU EVER TAKEN MUSIC LESSONS? (YES/NO): E2. IF YES, FOR
HOW MANY YEARS? (e.g., piano, 3 years; guitar, 5 years):
Instrument Years E3. IF YOU ARE NO LONGER TAKING LESSONS, HOW
LONG HAS IT BEEN SINCE YOUR LAST LESSON? (YEARS): F1. IF YOU PLAYED
AN INSTRUMENT AND/OR SANG IN THE PAST, WHAT WAS THE AVERAGE NUMBER
OF HOURS A WEEK THAT YOU PLAYED AND/OR SANG PRIOR TO HEARING LOSS?
(1) 1 – 3 hours; (2) 3 – 6 hours; (3) 6 – 9 hours; (4) 9 – 12
hours; (5) 12 hours or more. ANSWER: F2. HOW LONG DID YOU MAINTAIN
THIS INTENSITY OF PLAYING/SINGING? (YEARS): G1. IF YOU PLAYED AN
INSTRUMENT AND/OR SANG IN THE PAST, ON AVERAGE, HOW MANY HOURS A
WEEK DID YOU PLAY AND/OR SING AFTER HEARING LOSS BUT PRIOR TO
IMPLANT SURGERY? (1) 1 – 3 hours; (2) 3 – 6 hours; (3) 6 – 9 hours;
(4) 9 – 12 hours; (5) 12 hours or more. ANSWER: G2. HOW LONG DID
YOU MAINTAIN THIS RATE OF PLAYING/SINGING? (YEARS): H1. IF YOU
CURRENTLY PLAY AN INSTRUMENT AND/OR SING, HOW MANY HOURS A WEEK DO
YOU PLAY OR SING? (1) 1 – 3 hours; (2) 3 – 6 hours; (3) 6 – 9
hours; (4) 9 – 12 hours; (5) 12 hours or more. ANSWER: H2. HOW LONG
HAVE YOU MAINTAINED THIS RATE? (YEARS):
-
39
I. DID YOU PARTICIPATE IN MUSIC ENSEMBLES (CHOIR, BAND, INFORMAL
GROUP, ETC) PRIOR TO YOUR HEARING LOSS? (YES/NO): J. DID YOU
PARTICIPATE IN MUSIC ENSEMBLES (CHOIR, BAND, ETC) AFTER HEARING
LOSS BUT PRIOR TO IMPLANT SURGERY? (YES/NO): K. DO YOU PARTICIPATE
IN MUSIC ENSEMBLES (CHOIR, BAND, ETC) SINCE IMPLANT SURGERY?
(YES/NO): L. TYPES OF MUSIC ENJOYED PRIOR TO HEARING LOSS: (e.g.,
classical, pop, country) M. TYPES OF MUSIC ENJOYED AFTER HEARING
LOSS BUT PRIOR TO IMPLANT SURGERY: (e.g., classical, pop, country)
N. TYPES OF MUSIC ENJOYED SINCE IMPLANT SURGERY: (e.g., classical,
pop, country)
-
40
FOR THE FOLLOWING QUESTIONS, PLEASE INDICATE HOW MUCH THE
FOLLOWING STATEMENTS APPLY TO YOU. O. I ENJOYED LISTENING TO MUSIC
PRIOR TO MY HEARING LOSS. 1. Strongly disagree 2. Disagree 3.
Neither agree nor disagree 4. Agree 5. Strongly agree ANSWER: P. I
ENJOYED LISTENING TO MUSIC WITH MY HEARING AID(S). 1. Strongly
disagree 2. Disagree 3. Neither agree nor disagree 4. Agree 5.
Strongly agree ANSWER: Q. I ENJOY LISTENING TO MUSIC WITH MY
COCHLEAR IMPLANT(S). 1. Strongly disagree 2. Disagree 3. Neither
agree nor disagree 4. Agree 5. Strongly agree ANSWER: R. THE SOUND
OF MUSIC AS HEARD WITH MY COCHLEAR IMPLANT(S) IS PLEASANT. 1.
Strongly disagree 2. Disagree 3. Neither agree nor disagree 4.
Agree 5. Strongly agree ANSWER:
-
41
MUSIC & COCHLEAR IMPLANTS List of Familiar Musical
Excerpts
• Please indicate in the table below a list of 10 musical
selections that you listen to
regularly. If possible, select only one track from each album
and/or each artist. • If you listen to fewer than 10 musical tracks
from different artists or different
albums, provide whatever information you can in the table
below.
TRACK NAME ARTIST/PERFORMER ALBUM HOW LONG HAVE YOU BEEN
FAMILIAR WITH THIS
SELECTION? EXAMPLE: Clair de lune (from Suite bergamasque)
Claude Debussy / François-Joel Thiollier
Debussy: Piano Works Vol. 1
3 years
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
-
42
Appendix B
MUSIC BACKGROUND INFORMATION QUESTIONNAIRE (ADULTS) Name:
Gender: F M Student # Date of Birth : Age: Phone # Country of Birth
First language Musical Background Have you ever taken private music
lessons (instrumental or voice)? Yes No (Circle one) Years since
Years playing Years since last Instrument Years of lessons last
lessons regularly played regularly
Have you ever taken music in elementary or high school? Yes No
(Circle one) Years since Years since last Instrument Years of
lessons last lessons Years playing played regularly
Have you ever studied music theory? If so, how extensively?
(i.e., levels achieved or courses taken). How much ear training
have you had? Would you describe yourself as being "tone deaf"? Yes
No (Circle one) Do
you have absolute pitch (perfect pitch)? Yes No (Circle one)
-
43
Did you grow up listening to music primarily from Canada, the
United States, or England?
Yes No (Circle one) If not, which country's music did you grow
up listening to?
Please describe your musical activities (singing, playing,
dancing, listening, etc.) Regardless of whether you've ever taken
music lessons, please rate how "musical" you think you are in
relation to the average person.
extremely unmusical average extremely musical
1 2 3 4 5 6 7 Please describe your music listening: Type of
Music Hours per week listening
-
44
Appendix C
Music and Cochlear Implants Interview
Please answer these questions with a few sentences or a short
paragraph. You can answer in prose or in point-form. You are not
obligated to fill the entire space, but you can do so if you
wish.
1. How would you describe the role of music in your life (i.e.
is it important, not important, why, etc)? 2. Please describe to us
the way you enjoy music on a day-to-day basis (i.e. when and where
you listen to and / or play music, how it makes you feel, etc) 3.
What do you generally like (or not like) about music? Please
specify whether you are referring to playing or listening to music.
Feel free to refer to both of these activities in your answer.
-
45
Appendix D
SEMI-STRUCTURED Interview open-ended questions (prompts)
1. Perhaps you can tell me about the role of music in your life
to before your hearing loss. 2. Can you describe your experience
with music during the period when you used hearing aids? 3. How has
your experience of music changed since getting your cochlear
implant? 4. Are you still able to enjoy music? 5. How would you
describe the sound of music as heard through your implant? How
could you compare it to what you think is heard by someone with
normal hearing? 6. Are you still able to enjoy the specific musical
pieces that you liked before your hearing loss? 7. Have your
musical preferences changed since receiving your implant? 8. When
you listen to music, to which part do you primarily listen to? Was
this different before receiving your implant?