From Networked Music to Behavior Driven Interaction

From Networked Music to Behavior-‐Driven Interac9on

Álvaro Barbosa -‐ www.abarbosa.org

Research Center for Science and Technology of the Arts (CITAR) School of Arts, Catholic University Porto – www.artes.ucp.pt

Slide # 2 / Author: Álvaro Barbosa (www.abarbosa.org)

h=p://www.abarbosa.org/ Since 1998 – EA, UCP-‐Porto, Portugal

hIp://artes.ucp.pt/

Since 2004 – CITAR, UCP-‐Porto, Portugal

hIp://artes.ucp.pt/citar/

2000/2006 – MTG, UPF-‐Barcelona, Spain

hIp://mtg.upf.edu/

2010 – CCRMA, Stanford-‐CA, USA

hIp://ccrma.stanford.edu/

Since 2011 – USJ, Macau – China

hIp://www.usj.edu.mo/

Álvaro Barbosa


Álvaro Barbosa Very early Web Developer in the industry (Caleida & aeiou.pt, 1995-‐1998)


Research / Book

•  Displaced Soundscapes is a metaphorical description of the way lively generated sounds can be perceived over the Internet."

•  (Book) Barbosa, A. 2008. “Displaced Soundscapes”"•  VDM Verlag (ISBN: 978-3-8364-7154-1)"

-  Extensive Survey of NM Applications "-  Research on Latency and Internet Acoustics"-  Models for Networked Music Systems"-  Implementation “Public Sound Objects” "


Background on Network Music Performance

Performing Music at a Distance Synchronizing Berlin and Hong Kong (1/April/2011 – 8.753 Km)

Hong Kong New Music Ensemble & Ensemble Adapter

http://hknme.org/hongkongartsblog/?p=1525



Performing Music at a Distance



•  Stanford-Mcgill 2002 (4.076 Km)



•  Stanford-Mcgill 2002 – Discovery Channel



•  Stanford-Mcgill 2002 (the real thing)


Background on Network Music Performance Multiple Location – Same Performance ARTECH 2008: Rambouillet (FR) / Casa da Música (PT) / SARC (IR)

http://artes.ucp.pt/artech2008/



Latency and Networked Music

In a vocal conversation it is possible to maintain it even with one-way delays of up to 500 ms.

Holub, J., Kastner, M., Tomiska, O. 2007

In order to maintain a synchronized and smooth musical interaction reduces drastically to the order of tens of milliseconds.

Schuett, N. 2002 Chafe C., Gurevich M, 2004 Lago, N and Kon, F. 2004 Barbosa, A., Carôt, A. 2005 Chew, E., Sawchuk, A., Tanoue, C., and Zimmermann, R. 2005 Bartlette, C., Headlam, D., Bocko, M., Velikic, G. 2006 Farner, S., Solvang, A., Sæbo, A., Svensson, U. P. 2009 Chafe, C., Cáceres, J-P., Gurevich, M., 2010




For the Human ear to perceive the order of two simultaneous sounds, they should not be displaced in time over 20ms (Hirsh, 1959)

A difficulty in discerning the order of sounds events

ê hard to maintain a synchronous musical interaction.



•  Jitter and delay asymmetry introduce further disruption in Network Communication.

•  Furthermore, Latency is not a technological condition that can be overcome in the near future. Consider mobile or satellite technology or a communication setup that spans worldwide.

Moscow – Santiago (fiber-optic link): Bidirectional Latency= 94,3 ms

Unidirectional Latency= 47,15 ms




The ability to perform music synchronously is strongly dependent on:

•  The music expressive qualities (Dynamics and Articulation)

•  The music style (rhythm, melody, harmony)

•  The music perceptual qualities (pitch, texture, timbre)

•  The music structure/form

•  The musician’s experience and practicing strategies

•  Complementary feed-back modalities (visual, tactile)

•  The listening conditions



Barbosa, A. 2003. “Displaced Soundscapes: A Survey of Network Systems for Music and Sonic Art Crea9on” Leonardo Music Journal 13, pp. 53-‐59 -‐ MIT Press, Cambridge; doi:10.1162/096112104322750791).




Basic Principles - Remote Performance (1) Ensemble Performance Threshold (EPT)

(2) Echo Feed-Back (Self Delay)

(3) Inverse Proportion to Tempo

(4) Reverb and Complementary Modalities

(5) Slow Attack Times

(6) Antiphony



(1) Ensemble Performance Threshold (EPT)

(CCRMA 2004, USC 2005)

DELAY (ms) 0 5 10 15 20 25 30 35 40 45 50 75 100 150

Difficult toCompensate

Comfort Zone for Untrained Users

EPT Interval forUntrained Users

Comfort Zone for Expert Musicians EPT Interval for Expert Musicians

ExtremelyDifficult to

Compensate

AlmostImpossible toCompensateUnnaturally

Low Latency



(2) Echo Feed-Back (USC 2005)

Chew, E., Sawchuk, A., Tanoue, C., and Zimmermann, R. 2005



(2) Echo Feed-Back (UCP-Porto 2005)



(3) Inverse Proportion to Tempo (UCP-Porto 2005)



(3) Inverse Proportion to Tempo (CCRMA/Banff 2006)

St. Lawrence String Quartet

(over 50ms Delay)



(4) Reverb and Complementary Modalities

Studies on the effect of Reverberation (Farner 2009) The Influence of Delay and Various Acoustic Environments Studies on the effect of Visual Feed-Back Pilot experiments at CCRMA since 2006 Studies on the effect of Haptic Feedback ???



(5) Slow Attack Times

Bregman (1990) Auditory Scene Analysis

This experiment clearly reveals that it is harder to perceive the order of overlapping sound events when their rise ]me (aIack) is slower. In other words, overlapping sounds with slower rise ]mes are beIer perceived as synchronous even when their onsets are not physically simultaneous.



•  Pilot Study Hypothesis

•  If slow a?ack @mes allow a be?er percep@on of sound simultaneity this might also lead to a be?er ability to perform synchronous musical interac@on

•  Experimental Setup



•  Rhythmic Performance •  80 BPM (to avoid possible overlaying of subsequent notes) •  Slow Tempo => Higher EPT •  Range from 25 ms to 110 ms (with steps of 10 ms) •  28 trials for Slow and Fast aIack strokes



Results How each musician performs in rela@on to the other? The analysis was performed using the soeware tool MATCH: Music Alignment Tool Chest, developed by Simon Dixon (this soeware analyses the alignment of audio files using the OLTW algorithm)



(6) ANTIPHONY Technique of alternate or responsive singing by a choir in two separate sec@ons. Each sec]on sings alternate musical phrases

Or POLYCHORAL ANTIPHONY (AKA Vene]an polychoral style) When two or more groups of singers sing in alterna]on. Origin in the late Renaissance and early Baroque

Can this Technique improve network latency tolerance?




Basic Principles - Shared Sonic Environments (1) Digital Control Interface (software or tangible)

(2) Local Synthesis & Transmission of Control Data

(3) Peer-To-Peer Communication

(4) Latency Adaptive Dynamics

(5) Behavior Driven Interaction (Loose Coupling)




Examples Shared Sonic Environments THE HORGIE: COLLABORATIVE

ONLINE SYNTHESIZER by Jorge Herrera

PUBLIC SOUND OBJECTS by Álvaro Barbosa


Public Sound Objects

Public Sound Objects 2004/2006, MTG (BCN)



Local Network of PSOs – CITAR (Porto) commissioned by “Casa da Musica” http://en.wikipedia.org/wiki/Casa_da_musica









Permanent Installation Casa da Música (Renaissance Hall)


Behavior-‐Driven Interac9on

Small Fish (Fujihata e Furukawa 1999)



Radial String Chimes (1st prototype Developed at CCRMA – Stanford University) Interactive musical device triggered by motion applied to a spinning vinyl record. Users can spin the record, making hanging coffee straws to bounce and pluck twelve radial guitar strings, applied to a round wood board.





Used in Live Performances: Live Set at Galerija Kapelica in Ljubljana, Slovenia During The EARZOM Fes]val 2010



Jorney to the Last Fron9er -‐ with Victor Gama – B&W 2012



Live Set: 2 Ipads and 1 Iphone runing CS-‐Grain, Reactable and Cur]s; 1 Radio String Chimes; 2 Wind Blowers; 1 E-‐Bow



Victor Gama: www.victorgama.org Performed on His own Instruments – Dino, Acrux and Thoa



Audio Materials: Recorded at the Antarc]ca Expedi]onin January 2012 by Álvaro Barbosa and Victor Gama



Field Recordings at the Antarc9ca Peninsula



Video and Photography for Mul9media Perfoamances





Photography Books: Drieing Ice; Antarc]cans


School of Arts – Porto, Portugal

Thank You!!!!

[email protected]

[email protected] | hIp://www.abarbosa.org

From Networked Music to Behavior Driven Interaction

Education

lvaro barbosa h

music structureform

book barbosa

ms slide

music style rhythm

distance slide

performance artech

ptartech2008 slide