E XPLORING TACTILE MAPPINGS OF AUDITORY STIMULI AS A STEP TOWARD MAKING MUSIC MORE ACCESSIBLE FOR THOSE THAT ARE DEAF OR HEARING IMPAIRED . Feel the Music PROTOTYPE 2 EXPERIMENT 1 A FORMAL EXPERIMENT WAS CONDUCTED TO INVESTIGATE OUR EMOTIONAL RESPONSES TO TACTILE STIMULI AND IDENTIFY WHICH FEATURES OF HAPTIC SENSATIONS AFFECT THE USER ' S RESPONSE . RESULTS STIMULI • 42 S TIMULI WERE SINE AND SQUARE WAVES WITH VARYING FREQUENCY AND AMPLITUDE . • 60 STIMULI WERE GENERATED BY MULTIPLYING A 64H Z SINE WAVE WITH AN ENVELOPE WAVEFORM DEFINED BY A PA - RAMETER α. • 6 REPEATED STIMULI . • 20 PARTICIPANTS TOOK PART . • 108 TACTILE STIMULI WERE PRESENTED . • S TIMULI WERE HAPTIC SENSATIONS GENERATED BY A TACTOR PLACED ON EACH PARTICIPANT ’ S FINGERTIP . P ARTICIPANTS RATED THE STIMULI BY A ROUSAL ( INTENSITY ) AND V ALENCE ( PLEASANTNESS ). T HESE MEASURES ARE BASED ON THE C IRCUMPLEX M ODEL OF A FFECT , A WIDELY USED MODEL IN PSYCHOLOGY RESEARCH TO MEASURE EMOTIONAL RESPONSES . E XAMPLE STIMULI P ROGRESSION OF PROTOTYPES T ACTORS α VALUES FOR ENVELOPE STIMULI RESULTS T HE RESULTS LED TO THE CREATION OF A WEARABLE PROTOTYPE THAT EMPLOYED TACTILE SENSATIONS AS A MUSICAL INTERFACE . M APPINGS FROM MUSIC SAMPLES ONTO THE DEVICE WERE DESIGNED AND TESTED IN A SECOND EXPERIMENT . BY : ALICE HAYNES SUPERVISORS : DR J. LAWRY, DR J. ROSSITER E XAMPLE T ACTILE M APPING FOR “A SHES TO A SHES ’ A FTER MULTIPLE ITERATIONS AND MODIFICATIONS THE FINAL PROTO - TYPE CREATED WAS A SLEEVE CONTAINING : • 8 VIBRATION MOTORS WITH SMOOTH CASINGS PLACED IN SEQUENCE ALONG THE FOREARM . • A SQUEEZER MODULE FORMED OF AN ENCLOSED POCKET FILLED WITH AIR VIA A VACUUM PUMP ON THE UPPER ARM . T HESE WERE CONTROLLED BY AN A RDUINO L EONARDO WITH 3 STACKED MOTOR SHIELDS AND 1 SD CARD SHIELD ONTO WHICH MAP - PINGS WERE SAVED . T ACTILE MAPPINGS WERE PLAYED IN SYNC WITH MUSIC BY SENDING A START COMMAND VIA SERIAL FROM MATLAB AND PLAYING THE MUSIC SIMULTANEOUSLY . MAPPINGS C HOSEN MUSIC SAMPLES WERE CONVERTED INTO MIDI FILES FROM WHICH THE KEY MELODIES AND RHYTHMS WERE IDENTIFIED AND TRANSLATED INTO AN ARRAY OF INTENSITIES TO CONTROL THE MOTORS AND SQUEEZER OVER TIME . 7 MUSIC SAMPLES WERE USED , INCLUDING THE J AWS THEMTUNE , ‘I S THIS LOVE ’ B OB M ARLEY AND ‘A SHES TO ASHES ’ D AVID B OWIE . TESTING • 20 PARTICIPANTS TOOK PART . • 28 STIMULI WERE PRESENTED . • S TIMULI INCLUDED MUSIC SAMPLES , PURE TACTILE SENSATIONS AND SYNCRONISED COMBINATIONS OF THE TWO . P ARTICIPANTS RATED STIMULI BY V ALENCE ( PLEASANTNESS ) AND S E - LECTED ADJECTIVES FROM A LIST TAKEN FROM H ENVER ’ S A DJECTIVE C IRCLE TO DESCRIBE THE SENSATIONS . • M EAN V ALENCE RESPONSES ACROSS ALL PARTICIPANTS WERE STA - TISTICALLY THE SAME FOR M USIC ONLY STIMULI AND C OMBINATION STIMULI . (F) • M EAN V ALENCE RESPONSES TO T ACTILE ONLY STIMULI WERE STA - TISTICALLY LOWER THAN FOR THE STIMULI INCLUDING MUSIC . • P ER INDIVIDUAL SONG SAMPLE PREFERENCE FOR M USIC , T ACTILE OR C OMBINRTION PRESENTATIONS VARIED . (E) • O VER 60% OF RESPONSES PER SONG SAMPLE WERE STATISTICALLY SIMILAR FOR M USIC ONLY AND T ACTILE ONLY STIMULI SUGGESTING THAT MAPPINGS WERE EFFECTIVE . • A GE OF PARTICIPANTS DID NOT AFFECT THEIR RESPONSE TO MUSIC , BUT OLDER PARTICIPANTS GAVE STATISTICALLY LOWER V ALENCE RANK - INGS TO T ACTILE STIMULI THAN YOUNGER PARTICIPANTS . T HIS MAY BE DUE TO CHANGE IN SKIN SENSITIVITY WITH AGE . • A DJECTIVE RESPONSES GENERALLY MATCHED FOR M USIC AND T AC - TILE MAPPINGS BUT ALSO FOR PURPOSEFULLY NON - MATCHED STIMULI . I N CONCLUSION , THE PROTOTYPE SUCCESSFULLY DEMON - STRATED THE POTENTIAL FOR TACTILE STIMULATION TO BE USED AS A SENSORY SUBSTITUTE FOR ACOUSTIC MUSIC . • T HE EXPERIMENT FOUND THAT THERE IS A GENERAL NEGA - TIVE TREND BETWEEN A ROUSAL AND V ALENCE RESPONSES AS DEMONSTRATED BY THE MEAN RESPONSES ACROSS STIMULI . (A) • F REQUENCY HAS A SIGNIFICANT POSITIVE CORRELATION WITH ELICITED A ROUSAL RESPONSES . (B) • F REQUENCY IS SIGNIFICANTLY CORRELATED WITH 80% OF PARTICIPANT V ALENCE RESPONSES . H OWEVER , 6 WERE POSI - TIVE AND 10 WERE NEGATIVE . (B) • A MPLITUDE IS NOT SIGNIFICANTLY CORRELATED WITH A ROUSAL OR V ALENCE ALTHOUGH THERE IS A GENERAL POSI - TIVE CORRELATION WITH A ROUSAL AND NEGATIVE CORRELA - TION WITH V ALENCE . (D) • S QUARE WAVE STIMULI MORE AROUSING THAN S INE WAVE STIMULI WHICH MAY BE DUE TO THE POSITIVE CORRELATION BE - TWEEN ENERGY IN THE STIMULUS SIGNAL AND ELICITED A ROUSAL RESPONSE . • E NERGY IN A STIMULUS SIGNAL ALSO NEGATIVELY CORRE - LATED WITH V ALENCE RESPONSES . • R EPEATED STIMULI HAD A MEAN SPREAD OF RESPONSES ACROSS 15.5% OF THE SPACE PER PARTICIPANT . T HIS INDI - CATED RELIABILITY IN PARTICIPANT RESPONSES SINCE TESTS RUN FOR MULTIPLE RANDOM RESPONSES HAD A SPREAD OF 98%. (C) I N CONCLUSION , FREQUENCY , WAVEFORM SHAPE AND SIGNAL ENERGY HAVE A SIGNIFICANT EFFECT ON AF - FECTIVE RESPONSE . T HIS CAN BE GENERALISED ACROSS PARTICIPANTS FOR A ROUSAL , WHEREAS V A - LENCE IS MORE SUBJECTIVE TO THE INDIVIDUAL . P ARTICIPANTS WERE RELIABLY ABLE TO DIFFERENTIATE BETWEEN STIMULI . H OWEVER , RESPONSES VARIED DRAMATICALLY BETWEEN INDIVIDUALS FOR EACH SPE - CIFIC STIMULUS . Participant Number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Correlation Coefficient -1 -0.5 0 0.5 1 Arousal vs log 2 (Frequency) Valence vs log 2 (Frequency) Valence -1 -0.5 0 0.5 1 Arousal -1 -0.5 0 0.5 1 Participant Number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Correlation Coefficient -0.5 0 0.5 Arousal vs Amplitude Valence vs Amplitude Valence -1 -0.5 0 0.5 1 Arousal -1 -0.5 0 0.5 1 Time (s) 0 0.5 1 1.5 Amplitude (V) -4 -2 0 2 4 Time (s) 0 0.5 1 1.5 Amplitude (V) -4 -2 0 2 4 Time (s) 0 0.5 1 Amplitude (V) -1 -0.5 0 0.5 1 њ = [0, -1.5, 7, 1.5] Time (s) 0 0.5 1 Amplitude (V) -1 -0.5 0 0.5 1 њ = [7, -7, 1.5, 0] C IRCUMPLEX M ODEL OF A FFECT VALENCE AROUSAL Time (s) 0 0.5 1 1.5 2 Envelope amplitude (V) -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 њ = -7 њ = -1.5 њ = 0 њ = 1.5 њ = 7 Stimulus Number 1 2 3 4 5 6 7 Mean response across participants -1 -0.5 0 0.5 1 Music Tactile Combination Participant Number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Mean response across stimuli types -1 -0.5 0 0.5 1 Music Tactile Combination A F E B C D 0 50 100 150 200 250 300 time (sec) -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 amplitude 0 5 10 15 20 25 30 35 40 1 2 3 4 5 6 7 8 10 Motor number 0 5 10 15 20 25 30 35 40 time (sec) 40 45 50 55 60 65 70 75 80 85 note number R AW AUDIO FILE M IDI FILE