Eng. 100: Music Signal Processing DSP Lecture 11 DSP ...web.eecs.umich.edu/~fessler/course/100/l/l11-dsp.pdf · Eng. 100: Music Signal Processing DSP Lecture 11 DSP topics / P3 help

Eng. 100: Music Signal Processing

DSP Lecture 11

DSP topics / P3 help

Curiosity: http://www.wired.com/2014/08/gyroscope-listening-hackCuriosity: https://www.youtube.com/watch?v=hSCObIXDCJc

Announcements: Course evaluations: email receipt to [email protected] Final Exam: Thu. Dec. 17, 4-6 PM, 1500 EECS

1

http://www.wired.com/2014/08/gyroscope-listening-hackhttps://www.youtube.com/watch?v=hSCObIXDCJc

Outline

Part 1. P3 logistics Part 2. Finish BPM from last lecture Part 3. Filtering (by request) Part 4. Digital signals and quantization Part 5. Pitch and tempo shifting Part 6. Auto-tune Part 7. P3 help

2

Part 1. P3 logistics

3

P3 presentation schedule

Each team should prepare 10 minutes of talkingplus up to 5 minutes of integrated live demonstration.Total presentation should not exceed 15 minutes!

3-5 minutes of Q/A and transition Some of the live demonstration can be pre-recorded / playback,

but some of it must be truly live.

Tue Dec 8 lecture: 3-4 teamsThu Dec 10 lecture: 3-4 teamsPreferences? ??

First presentation should begin right at 10:40AM.Please come at 10:35AM to set up.Practice the audio/video in 1500 EECS beforehand

All students must attend all presentations during Tue/Thu lectures.

4

P3 presentation/report items

Read P3 DSP specifications for required components, including: spectrum and/or spectrogram (Hz!) additive synthesis demo GUI screen shot, transcriber screen shot error-rate vs SNR plot...

5

Part 2. BPM

6

Metronome signal

0 0.5 1 1.5 2 91

1

t [s]

x(t

)

play

% bpm1_gen

% generate metronome tick signal to test bpm estimator

S = 8192;

bpm = 120;

bps = bpm / 60; % beats per second

spb = 60 / bpm; % seconds per beat

t0 = 0.01; % each "tick" is this long

tt = 0:1/S:9; % 9 seconds of ticking

f = 440;

%x = 0.9 * cos(2*pi*440*tt) .* (mod(tt, spb) < t0); % tone

clf, subplot(211), rng(0)

x = randn(1,numel(tt)) .* (mod(tt, spb) < t0) / 4.5; % click via "envelope"

% sound(x, S)

% audiowrite('bpm1a.wav', x, S, 8)

7

Metronome BPM

0 0.5 1 1.5 2 2.5 3 3.5 40

10

20

30

40

50

60

shift [s]

au

toco

rre

latio

n

% bpm1_find

% first try at beats-per-minute (bpm) estimator

[x S] = audioread('bpm1a.wav');

x = x'; % row vector

N = numel(x);

%a = real(ifft(abs(fft(x,2*N)).^2)); % autocorrelation

a = real(ifft(abs(fft(abs(x),2*N)).^2)); % why abs?

spacing = [0:(2*N-1)]/S; % why?

good = (spacing > 60/300 & spacing < 60/25); % min and max reasonable bpm

clf, subplot(211)

plot(spacing, a, 'b.-', spacing, 10*good, 'g.-', spacing, a .* good, 'r.-')

xlabel 'shift [s]', ylabel 'autocorrelation', axis([0 4 0 60])

[~, index] = max(a .* good); % highest correlation for reasonable bpm range

disp(sprintf('estimated spb = %g, so bpm = %g', index/S, 60*S/index))

% ir_savefig -tight cw fig_bpm1b

8

BPM Summary

Is the preceding metronome signal periodic?

This small example illustrates several useful ideas. Noise blips Using modulo mod for repeating patterns Using logical operations like < to make binary signals. Constraining max to reasonable search range Looking for correlation between bursts of noisy signals using abs A few lines of Matlab code can do sophisticated DSP operations

Summary: (auto)correlation is quite widely useful

9

Part 3. Filtering

10

Music filtering example - spectra

0 1000 4000 22050

amplitude

0

0.06Spectrum of original signal x(t)

f [Hz]0 1000 4000 22050

amplitude

0

0.06Spectrum of filtered signal z(t)

play play

11

Music filtering example - ala Lab 3

% fig_filter1.m illustrate (low-pass) filtering[x, S] = audioread('../synth/cars.wav', [1 1e5]);x = x(:,1); % monoN = numel(x);fx = fft(x);cutoff_hz = [1000 4000];cutoff_index = round(cutoff_hz/S*N)fz = zeros(size(fx));keep = [(1+cutoff_index(1)):(1+cutoff_index(2))];fz(keep) = fx(keep);z = real(ifft(fz));

12

Part 4. Digital signals and quantization

13

Continuous-time signals and discrete-time signals

t [ms]0 2 4

x(t)

-1

0

1

Continuous-time = Analog signal

x(t) = cos(2 f t)

0 1761

0

1

Discretetime (sampled) signal

n [sample]

x[n

]

x[n] = cos(2 f n/S)

play

14

Digital signals and quantization

0 1761

0

1

Discretetime (sampled) signal

n [sample]

x[n

]

n [sample]0 176

y[n]

-1

0

1Quantization with 3 bits

x[n] originaly[n] quantized

play play

15

Spectrum of quantized signal (3 bits)

n [sample]0 176

y[n]

-1

0

1Quantization with 3 bits

x[n] originaly[n] quantized

f [Hz]0 500 1500 2500 3500 4500 5500 6500 7500

Spectrum

ofy[n]

0

1

16

Spectrum of quantized signal (6 bits)

n [sample]0 176

y[n]

-1

0

1Quantization with 6 bits

x[n] originaly[n] quantized

f [Hz]0 500 1500 2500 3500 4500 5500 6500 7500

Spectrum

ofy[n]

0

1

original: play 3 bit: play 6 bit: play 8 bit: playThe default for .wav files is 16 bits. (Accepts 8, 16, 24, or 32.)

17

Part 5. Pitch and tempo shifting

18

Time scaling via sampling rate

-1

0

1

0 8 16x[n]

n [sample]

Digital signal

-1

0

1

0 0.08 0.16

x(t)

t [s]

Analog signal: S = 100; sound(x, S)

-1

0

1

0 0.05 0.1 0.16

x(t)

t [s]

Analog signal: S = 160; sound(x, S)

Changing the sampling rate parameter: changes pitch and tempo.

sound(x,S) sound(x, S * 2^(6/12))play play

19

Tempo changes

How to play a recorded song faster or slower without changing pitch?

Phase vocoder [1] url [2] url [3] url [4] url [5] url url

Basic idea: Use short-time Fourier transform (STFT) to make spectrogram

(i.e., FFT of overlapping segments)

Modify spectrogram using interpolation, being careful with phase Synthesize signal from modified spectrogram using inverse STFT

(Inverse FFT via ifft of each segment, carefully combining.)

Example.

original: play 3/4 speed: play 3/2 speed: play

Note: can combine phase vocoder with sampling rate parameter change20

http://www.ee.columbia.edu/~dpwe/e6820/papers/FlanG66.pdfhttp://dx.doi.org/10.1109/TASSP.1976.1162810http://dx.doi.org/10.2307/3680093http://dx.doi.org/10.1109/ASPAA.1999.810857http://www.ee.columbia.edu/~dpwe/resources/matlab/pvoc/http://sethares.engr.wisc.edu/vocoders/matlabphasevocoder.html

Spectrograms

0

500

1000

3000

4000

0 1 2Hz

time [s]

Spectrum of original

0

750

1500

3000

0 1 2

Hz

time [s]

Spectrum of 1.5*S version

0

500

1000

3000

4000

0 1 2

Hz

time [s]

Spectrum of 1.5 times faster version

Using sound(x, 1.5*S) Using phase vocoder

original: playplay play

21

Part 6. Auto-tune

22

Auto-tune demo

play

play

Popularized by Cher (!) in 1998 hit Believe [wiki]http://www.mathworks.com/matlabcentral/fileexchange/26337-autotune-toy

tex/course/100-engin/demo/auto-tune-toy/AutoTuneToy.m

Recent example: https://www.youtube.com/watch?v=eq1FIvUHtt023

https://youtu.be/4p0chD8U8fA?t=40http://en.wikipedia.org/wiki/Auto-Tunehttp://www.mathworks.com/matlabcentral/fileexchange/26337-autotune-toyhttps://www.youtube.com/watch?v=eq1FIvUHtt0

Part 7. P3 help

Questions?

References

[1] J. L. Flanagan and R. M. Golden. Phase vocoder. Bell Syst. Tech. J., 45(9):1493509, November 1966.

[2] M. Portnoff. Implementation of the digital phase vocoder using the fast Fourier transform. IEEE Trans. Acoust. Sp. Sig. Proc., 24(3):2438,June 1976.

[3] M. Dolson. The phase vocoder: A tutorial. Computer Music Journal, 10(4):1427, 1986.

[4] J. Laroche and M. Dolson. New phase-vocoder techniques for pitch-shifting, harmonizing and other exotic effects. In IEEE Workshop onAppl. of Signal Processing to Audio and Acoustics, pages 914, 1999.

[5] D. P. W. Ellis. A phase vocoder in Matlab, 2002.

24

fd@rm@0: fd@rm@1: fd@rm@2: fd@rm@3: fd@rm@4: fd@rm@5: fd@rm@6: fd@rm@7: fd@rm@8: fd@rm@9: fd@rm@10: fd@rm@11: fd@rm@12: fd@rm@13: fd@rm@14: fd@rm@15: fd@rm@16: fd@rm@17: fd@rm@18: fd@rm@19: