8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
1/189
Helsinki University of Technology Laboratory of Acoustics and Audio Signal Processing
Espoo 1999 Report 53
VIRTUAL ACOUSTICS AND 3-D SOUND IN
MULTIMEDIA SIGNAL PROCESSING
Jyri Huopaniemi
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
2/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
3/189
Helsinki University of Technology Laboratory of Acoustics and Audio Signal Processing
Espoo 1999 Report 53
VIRTUAL ACOUSTICS AND 3-D SOUND IN
MULTIMEDIA SIGNAL PROCESSING
Jyri Huopaniemi
Dissertation for the degree of Doctor of Science in Technology to be presented with due permission for
public examination and debate in Auditorium S4, Department of Electrical and Communications
Engineering, Helsinki University of Technology (Espoo, Finland) on the 5th of November, 1999, at 12
o'clock noon.
Helsinki University of Technology
Deparment of Electrical and Communications Engineering
Laboratory of Acoustics and Audio Signal Processing
Teknillinen korkeakoulu
Shk- ja tietoliikennetekniikan osasto
Akustiikan ja nenksittelytekniikan laboratorio
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
4/189
Helsinki University of Technology
Laboratory of Acoustics and Audio Signal Processing
P.O.Box 3000
FIN-02015 HUT
Tel. +358 9 4511
Fax +358 9 460 224
E-mail [email protected]
Jyri Huopaniemi
Cover picture of MarienkircheErkki Rousku
ISBN 951-22-4706-2
ISSN 1456-6303
Libella Oy
Espoo, Finland 1999
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
5/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
6/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
7/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
8/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
9/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
10/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
11/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
12/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
13/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
14/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
15/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
16/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
17/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
18/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
19/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
20/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
21/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
22/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
23/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
24/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
25/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
26/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
27/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
28/189
and PropertiesSource
- source directivity
- modeling of
- artificial reverb
- modeling of
acoustic spaces
spatial hearing
Room Geometry
. speech and sound synthesis
MODELING
Multichannel
MEDIUM
SOURCE
RECEIVERRoom
DefinitionHRTF
Database
Listener
ModelingSource
Modeling
Modeling
.
REPRODUCTION
DEFINITION
Binaural
loudspeaker
absorption
HRTFs.. simple models
. propagation
headphone /
- natural audio
- synthetic audio
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
29/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
30/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
31/189
Hl Hr Hi Hi
Hc Hc
xm
y l y lyr yr
xl xr
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
32/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
33/189
DIRECTRAY-BASED
COMPUTATIONAL
MODELING
ELEMENT
METHODS
MODELING
MODELING OF ROOM ACOUSTICS
INDIRECT
MODELINGMODELING
WAVE-BASED
MODELING
STATISTICAL
ACOUSTIC-SCALE
MODELING
MODELING
METHOD
DIFFERENCE
METHODS
RAY-
TRACING
IMAGE-SOURCE
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
34/189
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 21
0.5
0
0.5
1
Amplitude
Direct Sound
Early Reflections (< 80100 ms)
Late Reverberation (RT60 ~2.0 s)
Left Channel
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 21
0.5
0
0.5
1
Time / s
Amplitude
Right Channel
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
35/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
36/189
RAY TRACINGIMAGE-SOURCE
AURALIZATION
DIFFERENCEMETHODMETHOD
NON-REAL-TIME ANALYSISREAL-TIME SYNTHESIS
ACOUSTICAL ATTRIBUTES
REVERBERATION
ARTIFICIAL LATEDIRECT SOUND AND
EARLY REFLECTIONS
OF THE ROOM
MEASUREMENTS
ROOM GEOMETRY
MATERIAL DATA
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
37/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
38/189
T0
TN
( )z
( )zlate reverberationunit, R
directionalfiltering(ITD+HRTF)
( )z
crosstalkcancelingC
0F ( )z
m
x ( )n
yr
ly
. . .
. . .
. . .
. . .
. . .
. . .
z
z( )1
absorption,air and materialsource directivity,
1/r attenuation
F ( )
sound input
zT ( )1
z zz
direct sound and early reflections
out(left)
out(right)
(optional)
binaural outputlate reverberation
( )
z
n( )
n( )
-d -d -d
FN
0 1 N
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
39/189
Physical modelingGuitar synthesisDouble bass synthesisFlute synthesis
Conductor gestureanalysis
Animation &visualization
User interface
Image sourcecalculation
Auralizationdirect sound and early reflectionsbinaural processing (HRTF)
diffuse late reverberation
AscensionMotionStar
DisplaySynchr
onizat
ion
Midic
ontrol
Instrument audio(ADAT, Nx8 channels)
loudspeakersor with
Motiondata
Listenermovements
Conductor Listener
MIDISynthesizerfor drums
Optional ext.audio input
Listenerposition data Binaural reproduction
either with headphones
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
40/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
41/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
42/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
43/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
44/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
45/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
46/189
1
2
^y (n)
M
Sound
source
D (z)
1
My (n)
y (n)1
y (n)2Sound y (n)2
My (n)
y (n)
sourceD (z)
D (z)
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
47/189
0
5
10
0
50
100150
-20
-15
-10
-5
0
Frequency (kHz)Azimuth Angle ()
Magnitude(dB)
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
48/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
49/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
50/189
102
103
104
050
100150
30
25
20
15
10
5
0
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
51/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
52/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
53/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
54/189
Directivity of BK4128 dummy head, mouth opening mic position
30
210
60
240
90
270
120
300
150
330
180 00 dB6 dB12 dB18 dB
125 Hz
250 Hz
500 Hz
2000 Hz
4000 Hz
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
55/189
Directivity of BK4128 dummy head, mouth transducer mic position
30
210
60
240
90
270
120
300
150
330
180 00 dB6 dB12 dB18 dB
125 Hz
250 Hz
500 Hz
2000 Hz
4000 Hz
Spherical head model directivity
30
210
60
240
90
270
120
300
150
330
180 00 dB6 dB12 dB18 dB
125 Hz
250 Hz
500 Hz
2000 Hz
4000 Hz
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
56/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
57/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
58/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
59/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
60/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
61/189
2
1
0
Material 1. Order: a=1, b=1
2
1
0
Material 2. Order: a=1, b=1
2
1
0
Material 3. Order: a=1, b=1
7
6
5
4
3
Material 4. Order: a=3, b=3
2
1
0
Material 5. Order: a=1, b=1
2
1
0
Material 6. Order: a=1, b=1
2
1
0
Material 7. Order: a=1, b=1
Magnitude(dB)
2
1
0
Material 8. Order: a=1, b=1
Magnitude(dB)
3
2
1
0
Material 9. Order: a=1, b=1
3
2
1
0
Material 10. Order: a=1, b=1
2
1
0
Material 11. Order: a=1, b=1
6
4
2
0
Material 12. Order: a=3, b=3
2
1
0
Material 13. Order: a=1, b=1
10
5
0
Material 14. Order: a=3, b=3
200 1000 10000
15
10
5
0
Material 15. Order: a=3, b=3
Frequency (Hz)
200 1000 10000
10
5
0
Material 16. Order: a=3, b=3
Frequency (Hz)
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
62/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
63/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
64/189
M
AI
material absorption
source directivity
air absorption
distance attenuation
0-N( )zT ={
0-N
1-Nz( )
z( )
D0-N
z( )
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
65/189
0-N( )zF
ITD0-N,L
( )z
minimum-phaseHRTFs
interaural timedifference
ITD0-N,R
( )z
={H
0-N,L( )z H
0-N,R( )z
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
66/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
67/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
68/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
69/189
USER INPUTUSER INPUTDADATTAA
HRTF Database = [0:10:350]
= [90:10:90]
AudioInput
ITDTable
Azimuth Elevation
CoefficientInterpolation
DLl
DLr
hl,i(n,,)
hr,i(n,,)
LoudspeakerListening
HeadphoneListening
OUTPUTOUTPUT
Cross-talk
Canceling
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
70/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
71/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
72/189
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
x 103
2
1
0
1
2
Impulse response, azim= 40, id=2
Amplitude/origi
nal
Left
Right
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
x 103
2
1
0
1
2
Time / s
Amplitude/min.p
hase
Left
Right
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
73/189
103
104
25
20
15
10
5
0
5
10
15
20
25Magnitude response, azim= 40, id=2
Frequency / Hz
Magnitude/dB
Left
Right
Original ROriginal LMin.phase RMin.phase L
A A'
asin
a
a
Left
Right
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
74/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
75/189
103
104
30
20
10
0IPD, azim= 40, id=2
Phase(unwrapped)
/rad
Original IPDMin.phase + linear excess approx.
103
104
1
0.8
0.6
0.4
0.2
0x 10
3 ITD, azim= 40, id=2
Frequency / Hz
ITD/s
Original ITDMin.phase + linear excess approx.LF model (01.5 kHz)HF model (1.520 kHz)
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
76/189
Frequency-independent modelKuhn low-freq modelKuhn high-freq modelMeasured ITDs (33 subjects)
0 50 100 150 200 250 300 350
-8
-6
-4
-2
0
2
4
6
8
x 10-4
Azimuth angle (deg)
ITD(s)
ITD approximations vs. measurements, a=0.0875 m
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
77/189
0 50 100 150 200 250 300 350
5
0
5
x 104
Azimuth (degrees)
ITD(s)
ITD measured at different elevations for a human subject
30 deg
15 deg
0 deg
60 deg
90 deg
0 50 100 150 200 250 300 350
5
0
5
x 104
Azimuth (degrees)
ITD(s)
Modeled elevationdependent ITD
30 deg
15 deg
0 deg
60 deg
90 deg
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
78/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
79/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
80/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
81/189
0 50 100 150 200 250 300 350
5
0
5
x 104 ITD for distances 1.9 and 0.7 m
Azimuth (deg)
ITD(s)
HRTF Farfield
HRTF Nearfield
0 50 100 150 200 250 300 350
5
0
5
x 104
Azimuth (deg)
IT
D(s)
Spherical FarfieldSpherical Nearfield
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
82/189
102
103
104
-30
-25
-20
-15
-10
-5
0
5
10
Frequency (Hz)
MagnitudeResponse(dB)
Spherical head HRTF for a source at 0.3 m distance
0
50
70
90
100
110
130
140
150
180
170
102
103
104
-25
-20
-15
-10
-5
0
5
10
Frequency (Hz)
MagnitudeResponse(dB)
Spherical head HRTF for a source at 2.0 m distance
0
90
100
110
130
140
150
180
160
170
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
83/189
102
103
104
10
0
10
20
30
Magnitude(dB)
ILD comparison, mean across 9 subjects, azi=0 deg
HRTF Farfield
HRTF NearfieldSpherical FarfieldSpherical Nearfield
102
103
104
10
5
0
5
10
Frequency (Hz)
Magnitu
de(dB) HRTF Fartonearfield gain
Spherical Fartonearfield gain
102
103
104
10
0
10
20
30
Magnitude(dB
)
ILD comparison, mean across 9 subjects, azi=30 deg
102
103
104
10
5
0
5
10
Frequency (Hz)
Magnitude(dB)
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
84/189
102
103
104
10
0
10
20
30
Magnitude(dB)
ILD comparison, mean across 9 subjects, azi=60 deg
102
103
104
10
5
0
5
10
Frequency (Hz)
Magnitu
de(dB)
102
103
104
10
0
10
20
30
Magnitude(dB
)
ILD comparison, mean across 9 subjects, azi=120 deg
102
103
104
10
5
0
5
10
Frequency (Hz)
Magnitude(dB)
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
85/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
86/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
87/189
102
103
104
100
101
102
Frequency (Hz)
Resolution(Q-value)
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
88/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
89/189
102
103
104
80
70
60
50
40
30
20
10
0
10
Frequency (Hz)
RelativeMagnitude(dB)
HRTF: person: 1, azimuth: 0, elevation: 0, right ear
256tap origBark
ERB
1/3 oct
1/10 oct
Cepstral
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
90/189
0 10 20 30 40 50 60 70 80 906
5
4
3
2
1
0
1
Time (samples)
Amplitude
HRIR: person: 1, azimuth: 0, elevation: 0, right ear
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
91/189
102
103
104
100
101
102
Frequency (Hz)
RelativeWeight
ERB weightingBark weighting
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
92/189
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
=-0.8
=-0.
6
=-0.
4
=-0.2
=0.0
=0.2
=0.4
=0.6
=0.8
Normalized original frequency
Normalizedwarpedfrequen
cy
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
93/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
94/189
102
103
104
30
20
10
0
10
20
30
40
50
60
Frequency (Hz)
RelativeMagnitude(dB)
HRTF: person: 1, azimuth: 20, elevation: 0, right ear
OriginalDf equalized
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
95/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
96/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
97/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
98/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
99/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
100/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
101/189
0
1
2
etc.
in
out
(a)
z-1
0
1
2
z -1
z-1
etc.
in
out
+
+
(b)
D (z)1
D (z)1
x 0
x 1
x 2
z-1
z-1
z-1
1
2
0
1
2
etc.
ina) b)
out+
+
+
x 0
x 1
y 1
y 2
y 3
x 2
z -1
z -1
z -1
1
2
3
0
1
2
etc.
g=1/in out
+
+
+
0
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
102/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
103/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
104/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
105/189
0 01 1
01
0 0
0 0
1 1
1 1
0 0
0 0
1 1
1 1
01
h
D
C
Bh
h
0 0
0 0
1 1
1 1
Ah
Eh
0 0
0 0
1 1
1 1
0 0
0 0
1 1
1 1
0
0
1
1
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
106/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
107/189
noise
low passfiltering
GTFB
GTFB
rectificationhalf wave
Spectrum
Spectrum
Left LL
Right LL
spectrum
ITD
pink
HRTF
HRTFL
R
LL
LL
LL
IACC
IACC
IACC
LL
LL
LL
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
108/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
109/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
110/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
111/189
Minimum-phase OriginalFIR, window: boxcar, order: 40WFIR, lambda=0.65, window: boxcar, order: 13IIR, design: Prony, order: 20
WIIR, lambda= 0.65, design: Prony, order: 10WIIR2, lambda= 0.7233, design: Prony, order: 20 BMT IIR, order: 10
102
103
104
20
30
40
50
60
70
80
90
Frequency (Hz)
RelativeMagnitude(dB)
Modeling of DF-equalized minimum-phase KEMAR HRTFs: elev=0, azi=30
Minimum-phase OriginalFIR, window: boxcar, order: 80WFIR, lambda=0.65, window: boxcar, order: 27
IIR, design: Prony, order: 40WIIR, lambda= 0.65, design: Prony, order: 20WIIR2, lambda= 0.7233, design: Prony, order: 40
102
103
104
-40
-30
-20
-10
0
10
20
Frequency (Hz)
RelativeMagnitude(dB)
Modeling of minimum-phase B&K4100 HRTFs: elev=0, azi=30
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
112/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
113/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
114/189
102
103
104
-80
-70
-60
-50
-40
-30
-20
-10
0
Frequency (Hz)
RelativeMagnitude(dB)
IIR approximation, azimuth=135, elevation 0, left ear
Original 256-tap FIR
48
36
24
12
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
115/189
102
103
104
-80
-70
-60
-50
-40
-30
-20
-10
0
Frequency (Hz)
RelativeMagnitude(dB)
WIIR approximation, azimuth=135, elevation 0, left ear, lambda=0.65
Original 256-tap FIR
48
36
24
12
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
116/189
20 30 40 50 60 70 80 90 100 110 1200
2
4
6
8
10
12
14
16
18
20
Number of Filter Coefficients
SpectralDistanceMeasure
FIR
IIR
WIIR
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
117/189
1 2 3
10
20
30
40
50
60
70
80
90
Listening test results: azimuth angles 0, 135
FilterOrder
HRTF Approximation Type
FIR IIR WIIR
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
118/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
119/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
120/189
103
104
30
25
20
15
10
5
0
5
10
15
20
Frequency / Hz
RelativeMagnitude/dB
Modeling of minimumphase HRTFs: azi=40deg, person: 3
Minimumphase OriginalFIR, order: 48
IIR, design: Prony, order: 24WIIR, lambda= 0.65, design: Prony, order: 24
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
121/189
70
80
90
3.1908
FIR, azi=40, id=3
70
80
90
5.1938
70
80
90
5.4452
LL(L)/phons
7080
90
7.1978
0.2 1 3 10 21
70
80
90
9.0677
3.0824
IIR, azi=40, id=3
5.8484
7.1516
8.6572
0.2 1 3 10 21
9.7706
Frequency / kHz
0.96047
WIIR, azi=40, id=3
97
1.9814
65
2.433
49
4.4735
33
0.2 1 3 10 21
7.4997
17
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
122/189
70
80
90
3.1241
FIR, azi=40, id=3
70
80
90
5.0635
70
80
90
5.1856
LL(R)/phons
7080
90
6.3121
0.2 1 3 10 21
70
80
90
8.1712
2.4967
IIR, azi=40, id=3
5.293
6.2493
7.4215
0.2 1 3 10 21
8.2394
Frequency / kHz
0.7613
WIIR, azi=40, id=3
97
1.7185
65
2.1406
49
5.0108
33
0.2 1 3 10 21
7.3156
17
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
123/189
1
0.5
0FIR, azi=40, id=3
1
0.5
0
1
0.5
0
ITD/ms
1
0.5
0
0.2 0.5 11
0.5
0
IIR, azi=40, id=3
0.2 0.5 1Frequency / kHz
WIIR, azi=40, id=3
97
65
49
33
0.2 0.5 1
17
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
124/189
10 20 30 40 50 60 70 80 90 1002
3
4
5
6
7
8
Number of filter coefficients
Compositemodelingerror
Subjects: 9, azims: 4
FIRIIRWIIR
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
125/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
126/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
127/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
128/189
PERSON
9876543219
5%C
IPredictedValueforLOCALIZATION
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
TYPE
FIR
IIR
WIIR
PERSON
98765432195%C
IPredictedValueforTIMBRE
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
TYPE
FIR
IIR
WIIR
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
129/189
FILTSIZ
25797654933179
5%C
IPredictedValueforLOCALIZATION
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
TYPE
FIR
IIR
WIIR
FILTSIZ
257976549331795%C
IPredictedValueforTIMBRE
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
TYPE
FIR
IIR
WIIR
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
130/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
131/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
132/189
102
103
104
80
60
40
20
0
20
40
60
21.8778
21.5842
21.6527
21.3595
21.2569
19.696
Frequency (Hz)
RelativeMagnitude(dB)
Model: Yulewalk, order: 8, person: 1, azimuth: 20, elevation: 0, right ear
256tap orig
Bark
ERB
1/3 oct
1/10 oct
UnsmoothedWindowed FIR
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
133/189
102
103
104
80
60
40
20
0
20
40
60
14.2365
16.3287
16.3926
16.6194
17.9536
19.696
Frequency (Hz)
RelativeMagnitude(dB)
Model: BMT, order: 8, person: 1, azimuth: 20, elevation: 0, right ear
256tap orig
Bark
ERB
1/3 oct
1/10 oct
UnsmoothedWindowed FIR
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
134/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
135/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
136/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
137/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
138/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
139/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
140/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
141/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
142/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
143/189
Measured HRTF, subject: jyhuMinimum-phase reconstruction
Gardner approximationCooper approximation
Measured HRTF, subject: jyhuMinimum-phase reconstructionGardner approximationCooper approximation
103
104
-20
0
20Magnitude responses of 30cross-talk canceling filters 1/(Hi(z)+Hc(z))
Frequency (Hz)
Magnitude(dB)
103
104
-20
0
20Magnitude responses of 30cross-talk canceling filters 1/(Hi(z)-Hc(z))
Frequency (Hz)
Magnitude(dB)
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
144/189
Measured HRTF, subject: jyhuMinimum-phase reconstructionGardner approximationCooper approximation
Measured HRTF, subject: jyhu
Minimum-phase reconstructionGardner approximationCooper approximation
103
104
0
1
2
3
Frequency (Hz)
Groupdelay(m
s)
Group delay of 30cross-talk canceling filters 1/(Hi(z)+Hc(z))
103
104
0
1
2
3
Frequency (Hz)
Groupdelay(ms)
Group delay of 30cross-talk canceling filters 1/(Hi(z)-Hc(z))
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
145/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
146/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
147/189
xl
xr
B
A
xl
xr
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
148/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
149/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
150/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
151/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
152/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
153/189
xl
xr
xl xr-d =+
-
-
+
Hl
Hr
l = dxl ,p
r = dxr,p
xr,p
xl ,p
+
-
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
154/189
xr,p
xl ,p
+
-
-+
delay
delay
Leftplacement
filter
Rightplacement
filter
xl 0
xr0
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
155/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
156/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
157/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
158/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
159/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
160/189
All subjects
Magnitude mean
All subjectsMagnitude mean
102
103
104
-20
0
20Crosstalk canceling design for +-10, 1/(Hi(z)+Hc(z)), 33 subjects
Amplitude(dB
)
102
103
104
-20
0
20Crosstalk canceling design for +-10, 1/(Hi(z)-Hc(z)), 33 subjects
Frequency (Hz)
Amplitude(dB)
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
161/189
Test subject: jyhuCooper approximation
Gardner approximationGeneralized min-phase VL
Test subject: jyhuCooper approximationGardner approximationGeneralized min-phase VL
103
104
-20
0
20VL design 10-90, left filter, shuffler structure
Magnitude(d
B)
103
104
-20
0
20VL design 10-90, right filter, shuffler structure
Frequency (Hz)
Magnitude(dB)
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
162/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
163/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
164/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
165/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
166/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
167/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
168/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
169/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
170/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
171/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
172/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
173/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
174/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
175/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
176/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
177/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
178/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
179/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
180/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
181/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
182/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
183/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
184/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
185/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
186/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
187/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
188/189
8/10/2019 Virtual Acoustics and 3D Sound in Multimedia Signal Processing
189/189