Switched-on Yampa: Declarative programming of modular synthesizers

Switched-On Yampa

Switched-On YampaDeclarative Programming of Modular Synthesizers

George Giorgidze Henrik Nilsson

School of Computer ScienceUniversity of Nottingham

Tenth International Symposium onPractical Aspects of Declarative Languages

San Francisco, USAJanuary 8, 2008

Switched-On Yampa

Outline

1 Modular Synthesizers

2 Yampa

3 Practical Aspects

4 Examples

5 Real Music

6 Performance

7 Conclusions

Switched-On Yampa

Modular Synthesizers

Modular Synthesizer

Switched-On Yampa

Modular Synthesizers

Clavia Nord Modular

Switched-On Yampa

Modular Synthesizers

Clavia Nord Modular

Switched-On Yampa

Yampa

What is Yampa?

Domain-specific language embedded in Haskell for programminghybrid (mixed discrete-time and continuous-time) systems.

Yampa is based on the concepts of Functional ReactiveProgramming (FRP) [Elliott, Hudak, Nilsson, Peterson, ...].

Switched-On Yampa

Yampa

Key Concepts

Signals: time-varying values

Signal Functions: functions on signals

Switching between signal functions

Yampa programs are structured using arrows [Hughes, Paterson]

Switched-On Yampa

Yampa

Key Concepts

Signals: time-varying values

Signal Functions: functions on signals

Switching between signal functions

Yampa programs are structured using arrows [Hughes, Paterson]

Programming model:

Switched-On Yampa

Yampa

Key Concepts

Signals: time-varying values

Signal Functions: functions on signals

Switching between signal functions

Yampa programs are structured using arrows [Hughes, Paterson]

Programming model:

Switched-On Yampa

Yampa

Signals and Signal Functions

Intuition:

type Signal α ≈ Time → α

x :: Signal T1

y :: Signal T2

type SF α β ≈ Signal α→ Signal β

f :: SF T1 T2

Switched-On Yampa

Yampa

Programming with signal functions

In Yampa, systems are described by combining signal functions (formingnew signal functions).

A combinator can be defined that captures this idea:

(≫) :: SF a b → SF b c → SF a c

Switched-On Yampa

Yampa

Signal Function Combinators

arr :: (a→ b)→ SF a b

(&&&) :: SF a b → SF a c → SF a (b, c)

switch :: SF a (b, Event c)→ (c → SF a b)→ SF a b

Switched-On Yampa

Practical Aspects

Some Points

Music can be seen as a hybrid phenomenon.

Music is sound: continuous pressure waves in some medium such asair.

Switched-On Yampa

Practical Aspects

Some Points

Music can be seen as a hybrid phenomenon.

Music is sound: continuous pressure waves in some medium such asair.Musical performance has clear discrete aspects.

Musical scoresMIDI StandardHaskore [Hudak]

Thus it is interesting to explore a hybrid approach to programmingmusic and musical applications.

Switched-On Yampa

Practical Aspects

Some Other Points

Yampa’s programming model is very reminiscent of programmingmodular synthesizers.

Switched-On Yampa

Practical Aspects

Some Other Points

Yampa’s programming model is very reminiscent of programmingmodular synthesizers.

We apply declarative Embedded Domain Specific Language (EDSL)to practical problem.

Switched-On Yampa

Practical Aspects

Some Other Points

Yampa’s programming model is very reminiscent of programmingmodular synthesizers.

We apply declarative Embedded Domain Specific Language (EDSL)to practical problem.

Fun application!

MIDI Music playerReal-time synthesizerCan be used in educational context

Switched-On Yampa

Practical Aspects

What we have done so far

Framework for programming modular synthesizers in Yampa:

Sound-generating and sound-shaping modules

Switched-On Yampa

Practical Aspects

What we have done so far

Framework for programming modular synthesizers in Yampa:

Sound-generating and sound-shaping modules

Supporting infrastructure:

Reading MIDI files (musical scores)Reading SoundFont files (instrument definitions)Writing result as audio files (.wav)Rendering audio directly to soundcard in real-timeKeyboard driver with basic GUI to play music with your synthesizer

Switched-On Yampa

Examples

Simple Patch

Switched-On Yampa

Examples

Simple Sine Oscillator

s = sin(2πft) (1)

Switched-On Yampa

Examples

Simple Sine Oscillator

s = sin(2πft) (1)

time :: SF a Time

oscSine :: Frequency → SF a Sample

oscSine f = time ≫ arr (λt → sin (2 ∗ pi ∗ f ∗ t))

Switched-On Yampa

Examples

Dynamically Controllable Sine Oscillator

φ = 2π

t∫

0

f (τ)dτ (2)

s = sin(φ) (3)

Switched-On Yampa

Examples

Dynamically Controllable Sine Oscillator

φ = 2π

t∫

0

f (τ)dτ (2)

s = sin(φ) (3)

oscSine :: Frequency → SF CV Sample

oscSine f0 = proc cv → do

let f = f0 ∗ (2 ∗∗ cv)phi ← integral−≺ 2 ∗ pi ∗ f

returnA−≺ sin phi

Switched-On Yampa

Examples

Dynamically Controllable Sine Oscillator

φ = 2π

t∫

0

f (τ)dτ (2)

s = sin(φ) (3)

oscSine :: Frequency → SF CV Sample

oscSine f0 = proc cv → do

let f = f0 ∗ (2 ∗∗ cv)phi ← integral−≺ 2 ∗ pi ∗ f

returnA−≺ sin phi

oscSine f0 =arr (λcv → let f = f0 ∗ (2 ∗∗ cv) in 2 ∗ pi ∗ f )≫ integral

≫ arr (λphi → sin phi)

Switched-On Yampa

Examples

Hello World!

constant 0 ≫ oscSine 440

Switched-On Yampa

Examples

Vibrato

Switched-On Yampa

Examples

Vibrato

constant 0

≫ oscSine 5.0

≫ arr (∗0.05)≫ oscSine 440

Switched-On Yampa

Examples

50’s Sci Fi

Switched-On Yampa

Examples

50’s Sci Fi

sciFi :: SF () Sample

sciFi = proc ()→ do

und ← arr (∗0.2) ≪ oscSine 3.0−≺ 0

swp ← arr (+1.0) ≪ integral −≺ −0.25

audio ← oscSine 440 −≺ und + swp

returnA−≺ audio

Switched-On Yampa

Examples

Envelope Generator

Switched-On Yampa

Examples

Envelope Generator

envGen :: CV → [(Time, CV )]→ (Maybe Int)→ SF (Event ()) (CV , Event ())

envBell = envGen 0 [(0.05, 1), (1.5, 0)] Nothing

Switched-On Yampa

Examples

Bell

FM synthesis is a form of audio synthesis where the timbre of a simplewaveform is changed by frequency modulating it with a modulatingfrequency that is also in the audio range, resulting in a more complexwaveform and a different-sounding tone.

Switched-On Yampa

Examples

Bell

FM synthesis is a form of audio synthesis where the timbre of a simplewaveform is changed by frequency modulating it with a modulatingfrequency that is also in the audio range, resulting in a more complexwaveform and a different-sounding tone.

bell :: Frequency → SF () (Sample, Event ())bell f = proc ()→ do

m ← oscSine (2.33 ∗ f )−≺ 0

audio ← oscSine f −≺ 2.0 ∗m

(ampl , end)← envBell −≺ noEvent

returnA−≺ (audio ∗ ampl , end)

Switched-On Yampa

Real Music

MIDI Music

Parse MIDI file and obtain event source:

SF a (Event [Midi .Message ])

Obtain MIDI messages from keyboard and feed them to runningsignal function during reactimation.

Switched-On Yampa

Real Music

Monophonic Synthesizer

Read samples (instrument recordings) together with other synthesisparameters from SoundFont file into internal table.

Oscillator similar to sine oscillator, except sine function replaced bytable lookup and interpolation.

We obtain collections of monophonic syntehsizers

Switched-On Yampa

Real Music

Exploit Yampa’s switching capabilities to:

create and switch in a monophonic synthesizer instance is responseto each note on event;

switch out the instance in response to a corresponding note offevent.

Switched-On Yampa

Real Music

Polyphonic Synthesizer

pSwitchB :: Functor col ⇒

col (SF a b) -- Initial signal func. collection→ SF (a, col b) (Event c) -- Triggers change→ (col (SF a b)→ c → SF a (col b)) -- Performs change→ SF a (col b)

Even t ( )

S a m p l e

S a m p l e

S a m p l e

m i x e r

t r i g g e r C h a n g e

p e r f o r m C h a n g e

m o n o S y n t h

m o n o S y n t h

m o n o S y n t h

M i d i E v e n t

S y n t h S t a t e

( [ N o t e I d ] , [ M o n o S y n t h I d ] , S y n t h S t a t e )

( S a m p l e , E v e n t ( ) )

( S a m p l e , E v e n t ( ) )

( S a m p l e , E v e n t ( ) )

M i d i E v e n t

t r a c k e n d d e t e c t o r

M i d i E v e n t

S a m p l e

Switched-On Yampa

Performance

Some Benchmarks

Environment:

Intel Core 2 Duo Processor T7200 2.00 GHzGHC 6.8.1 (using ”-O2” optimisation option)

Rendering audio to file

Mozart - Rondo Alla Turca; (duration = 207 sec)22050 Hz MONO - synthesized audio was generated in 206 sec

Rendering audio to soundcard (real-time)

16000Hz MONO - 8 notes can be played simultaniously22050Hz MONO - 6 notes can be played simultaniously

Switched-On Yampa

Conclusions

Conclusions

Status:

proof-of-conceptbut decent performancealready usable to some extent

Switched-On Yampa

Conclusions

Conclusions

Status:

proof-of-conceptbut decent performancealready usable to some extent

We apply FRP and Yampa to domain that has not been traditionallyassociated with pure declarative programming.