Physical and perceptual evaluation of the Interaural Wiener Filter algorithm Simon Doclo 1, Thomas J. Klasen 1, Tim van den Bogaert 2, Marc Moonen 1,

Physical and perceptual evaluation of the Physical and perceptual evaluation of the

Interaural Wiener Filter algorithmInteraural Wiener Filter algorithm

Simon Doclo1, Thomas J. Klasen1, Tim van den Bogaert2,

Marc Moonen1, Jan Wouters2

1Dept. of Electrical Engineering (ESAT-SCD), KU Leuven, Belgium2Laboratory for Exp. ORL, KU Leuven, Belgium

IHCON, Aug 19 2006

Slides available at http://homes.esat.kuleuven.be/~doclo/presentations.html

http://homes.esat.kuleuven.be/~doclo/presentations.html

22

OverviewOverview

• Binaural hearing aids: noise reduction and preservation of binaural cues

• Overview of binaural noise reduction algorithms

• Binaural multi-channel Wiener filter:

o Estimate of speech component at both hearing aids

o Speech cues are preserved – noise cues may be distorted

• Preservation of binaural cues:

o Extension of cost function with ITD-ILD-ITF expressions

• Experimental results:

o Physical evaluation (SNR, ITD, ILD)

o Perceptual evaluation (SRT, localisation)

• Audio demonstration

33

PL PR

ITD

ILD L

R

P

P

• Binaural auditory cues:

o Interaural Time Difference (ITD) – Interaural Level Difference (ILD)

o Binaural cues, in addition to spectral and temporal cues, play an important role in binaural noise reduction and sound localization

Problem statementProblem statement

• Hearing impairment reduction of speech intelligibility in background noise

o Signal processing to selectively enhance useful speech signal

o Many hearing impaired are fitted with hearing aid at both ears

o Multiple microphones available: spectral + spatial processing

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experimental results

Audio demo

Conclusions

44


• Bilateral system:

o Independent processing of left and right hearing aid

Problem statement





Audio demo

Conclusions

55


• Bilateral system:

o Independent processing of left and right hearing aid

o Localisation cues are distorted

• Binaural system:

o Cooperation between left and right hearing aid (e.g. wireless link)

o Assumption: all microphone signals are available at the same timeObjectives/requirements for binaural

algorithm:

1. SNR improvement: noise reduction, limit speech distortion

2. Preservation of binaural cues (speech/noise) to exploit binaural hearing advantage

3. No assumption about position of speech source and microphones

Problem statement





Audio demo

Conclusions

[Van den Bogaert, 2006]

66

Binaural noise reduction Binaural noise reduction techniquestechniques

Problem statement





Audio demo

Conclusions

77


Problem statement





Audio demo

Conclusions

88


• Fixed beamforming: spatial selectivity + binaural speech cues

o Maximize directivity index while restricting speech ITD error

o Superdirective beamformer using HRTFS

[Desloge, 1997]

[Lotter, 2004]

low computational complexity

limited performance, known geometry, broadside array, only speech cues

[Desloge, 1997]

Problem statement





Audio demo

Conclusions

99


• CASA-based techniques

o Computation and application of (real-valued) binaural mask based on binaural and temporal/spectral cues

[Kollmeier, Peissig, Wittkop, Dong, Haykin]

perfect preservation of binaural cues of speech/noise component

mostly for 2 microphones, “spectral-subtraction”-like problems

[Wittkop, 2003]

Problem statement





Audio demo

Conclusions

1010

• Adaptive beamforming: based on GSC-structure

o Divide frequency spectrum: low-pass portion unaltered to preserve ITD cues, high-pass portion processed using GSC


[Welker, 1997]

preserves binaural cues to some extent

substantial reduction in noise reduction performance, known geometry

[Welker, 1997]

Problem statement





Audio demo

Conclusions

1111


• Binaural multi-channel Wiener filter

o MMSE estimate of speech component in microphone signal at both ears

[Doclo, Klasen, Wouters, Moonen]

speech cues are preserved, no assumptions about position of speech source and microphones

noise cues may be distorted

Extension of MWF :

preservation of binaural speech and noise cues without substantially compromising noise

reduction performance

Problem statement





Audio demo

Conclusions

1212

Design of hearing aid SP algorithm

requires some mathematics

but perceptual evaluation in a couple of minutes…

1313

Configuration and signalsConfiguration and signals

• Configuration: microphone array with M microphones at left and right hearing aid, communication between hearing aids

• Vector notation: ( ) ( ) ( ) Y X V

noise component

0, 0, 0, 0( ) = ( ) , 0) =( 1m m mVY X m M 00, 0,, 0( ) = ( )( 0) , = 1mm mY V m MX

speech component

Problem statement





Audio demo

Conclusions

0 0 1 1( ) = ( ) ( ), ( ) = ( ) ( )H HZ Z W Y W Y

• Use all microphone signals to compute output signal at both ears

1414

Overview of cost functionsOverview of cost functions

Multi-channel Wiener filter (MWF): MMSE estimate of speech component in microphone signal at both ears

trade-off noise reduction and speech

distortion

Speech-distortion weighted multi-channel Wiener filter

(SDW-MWF)[Doclo 2002, Spriet 2004]binaural cue

preservation of speech + noise

Partial estimation of noise component

[Klasen 2005]

Extension with ITD-ILD or Interaural Transfer

Function (ITF)

[Doclo 2005, Klasen 2006]

Problem statement





Audio demo

Conclusions

1515

• Binaural SDW-MWF: estimate of speech component in microphone signal at both ears (usually front microphone) + trade-off between noise reduction and speech distortion

Binaural multi-channel Wiener Binaural multi-channel Wiener filterfilter

0

1

= , = ,x v M xx y v

M x v x

R R 0 rR r R R R

0 R R r

0

1

2 2

0, 0 0

11, 1

( )H H

r

HHr

XJ E

X

W X W VW

W VW X1=SDWW R r

trade-off parameter

Problem statement





Audio demo

Conclusions

estimate

o Depends on second-order statistics of speech and noise

o Estimate Ry during speech-dominated time-frequency segments, estimate Rv during noise-dominated segments, requiring robust voice activity detection (VAD) mechanism

o No assumptions about positions of microphones and sources

1616

Binaural multi-channel Wiener Binaural multi-channel Wiener filterfilter

• Binaural cues (ITD-ILD) :

Perfectly preserves binaural cues of speech component

Binaural cues of noise component speech component !!(cf. physical and perceptual evaluation)

Problem statement





Audio demo

Conclusions

• Extension of SDW-MWF with binaural cues

o Add term related to binaural cues of noise (and speech) component to SDW cost function

o Possible cues: ITD, ILD, Interaural Transfer Function (ITF)

o Weight factors and can be frequency-dependent

( ) = ( ) ( ) ( )x vtot SDW cue cueJ J J J W W W W

1717

Interaural Wiener FilterInteraural Wiener Filter

Problem statement





Audio demo

Conclusions

• Preserve binaural cues between input and output

o ITD: phase of cross-correlation

o ILD: power ratio

o ITF: Interaural transfer function (incorporates ITD and ILD)

0 0

1 1

Hv vout H

v

ZITF

Z

W V

W V

0 10

1 1 1

*0, 1,0, 0 1

*1, 1 11, 1,

( , )

( , )

r rrv vin

r vr r

E V VV r rITF

V r rE V V

R

Re.g.

0

1

2 2

0, 0 0

11, 1

2 2

0 1 0 1

( ) =H H

r

tot HHr

H x H H v Hin in

XJ E

X

E ITF E ITF

W X W VW

W VW X

W X W X W V W V

ITF preservation speech ITF preservation noise

o Closed form expression!

o large changes direction of speech component to noise component increase weight (cf. physical and perceptual evaluation)

1818

Overview of batch algorithmOverview of batch algorithmLeft input

signalsRight input

signals

( ) ( ) ( ) Y X VFFT FFT

0 0 0x vZ Z Z 1 1 1x vZ Z Z Left output Right output

IFFT IFFT

Frequency-domain filtering

Off-line computation of

statistics

VAD

( ), ( )v x R R

Calculate binaural input cues and

filter 0

1

( )( ) =

( )

WW

W, ,

Problem statement





Audio demo

Conclusions

1919

Experimental resultsExperimental results

• Identification of HRTFs:

o Binaural recordings on CORTEX MK2 artificial head

o 2 omni-directional microphones on each hearing aid (d=1cm)

o LS = -90:15:90, 90:30:270, 1m from head

o Conditions: T60=140 ms, fs=16 kHz, L=1366 taps

Problem statement




Experimental results- Physical- Perceptual

Audio demo

Conclusions

2020

Experimental resultsExperimental results

• Speech and noise material:

o Dutch sentences (VU list)

o Stationary speech-weighted noise with same long-term spectrum as speech material spatial aspects

o S0N60 , SNR=0 dB

o fs=16 kHz, FFT-size N=256, =1

• Physical evaluation:

o Speech intelligibility: SNR

o Localisation: ITD / ILD

• Perceptual evaluation:

o Preliminary study

o Speech intelligibility: SRT

o Localisation: localise S and N

Problem statement





Audio demo

Conclusions

2121

Physical evaluationPhysical evaluation

• Performance measures:

o Intelligibility weighted SNR improvement (left/right)

o ILD error (speech/noise component) power ratio

x xx out i in i

i

ILD ILD ILD

o ITD error (speech/noise component) phase of cross-correlation

x i x ii

ITD I ITD 1

* *0, 1, 0 10

{ } { }x i r r x xITD E X X E Z Z

L i L ii

SNR I SNR

importance of i-th frequencyfor speech intelligibility

low-pass filter 1500 Hz

Problem statement





Audio demo

Conclusions

Physical evaluation: SNRPhysical evaluation: SNR

00.05

0.10.15

0.2

00.1

0.20.3

0.40.5

0

5

10

15

20

25

SNR improvement left ear

SN

Rw [d

B]

00.05

0.10.15

0.2

00.1

0.20.3

0.40.5

0

5

10

15

20

25

SNR improvement right ear

S

NR

w [d

B]

Physical evaluation: ILD-ITDPhysical evaluation: ILD-ITD

00.05

0.10.15

0.2

0

0.2

0.4

0

5

10

15

ILD error speech component

IL

D [

dB]

00.05

0.10.15

0.2

0

0.2

0.4

0

5

10

15

ILD error noise component

IL

D [

dB]

00.05

0.10.15

0.2

0

0.2

0.4

0

0.5

1

1.5

ITD error speech component

IT

D [

rad]

00.05

0.10.15

0.2

0

0.2

0.4

0

0.5

1

1.5

ITD error noise component

IT

D [

rad]

2424

Physical evaluationPhysical evaluation

• Conclusions: increases: ITD-ILD error of noise component decreases

… BUT… ITD-ILD error of speech component increases

increases: ITD-ILD error of speech component decreases

… BUT… ITD-ILD error of noise component increases

o Compromise between speech and noise localisation error possible (cf. localisation experiments)

o SNR improvement only slightly degraded (cf. SRT experiments)

Problem statement





Audio demo

Conclusions

2525

• Speech intelligibility: SRT

o How does parameter affect speech intelligibility ?

o Two effects: increasing reduces SNR improvement, but preserves binaural noise cues better, enabling binaural speech intelligibility advantage

• Localisation performance

o How do parameters and affect localisation of processed speech and noise components ?

: preservation of speech cues, : preservation of noise cues

Perceptual evaluationPerceptual evaluation

Problem statement





Audio demo

Conclusions

2626

• Measurement procedure:

o SRT = SNR where 50% of speech is intelligible

o adaptive procedure (2 dB/step)

o headphone experiments, using HRTFs

o S0N60 (Dutch VU sentences – stationary noise)

o presentation level = 65 dB SPL

o 5 normal-hearing subjects

o fs=16 kHz, FFT-size N=256, =1, =0

o Reference condition = no processing

Perceptual evaluation: SRTPerceptual evaluation: SRT

HRTFx

HRTFv

speech

noise

GBinaural

filter

Mic L

R

Headphones

Problem statement





Audio demo

Conclusions

2727

Perceptual evaluation: SRTPerceptual evaluation: SRT

VU noise 60 deg, alpha=0

9,00

11,00

13,00

15,00

0,0 0,1 0,3 1,0 10,0

Beta

SR

T im

pro

vem

ent

• Results:

o average SRT without processing = -9.2 dB

o SRT improvements in the range 11-13 dB

o Binaural speech intelligibility advantage does not seem to compensate for loss in SNR improvement

Problem statement





Audio demo

Conclusions

2828

• Sum of localisation errors Sx and N0

• Parameters can be tuned to achieve better overal localization performance at the cost of some noise reduction

• Good correlation between physical and perceptual evaluation

Perceptual evaluation: Perceptual evaluation: localisationlocalisation

Loc error Sx + Loc error N0 5 subjects SxN0

0

10

20

30

40

50

60

70

80

0 0,1 0,3 1 10 100

beta

(°)

a l p h a = 0

a l p h a = 0 , 5

Problem statement





Audio demo

Conclusions

2929

Audio demonstrationAudio demonstration

• Speech and noise material:

o HINT sentences, speech source in front (0)

o Multi-talker babble noise at 60

o SNR=0 dB, fs=16 kHz, FFT-size N=256, =1, =0

Noisy Speech Noise

Input

Output (=0)

Output (=0.05)

Output (=10)

Problem statement





Audio demo

Conclusions

3030

• Speech enhancement for binaural hearing aids:

o Improve speech intelligibility

o Localisation: preserve binaural speech and noise cues

o No assumptions about position speech source and microphones

• Suitable algorithm: multi-channel Wiener filter

speech cues are preserved noise cues may be distorted

• Preservation of binaural noise cues:

Interaural Wiener filter: extension with Interaural Transfer Function of noise (and speech) component

• Perceptual evaluation:

o S0N60: SRT improvements in the range 11-13 dB

o Binaural speech intelligibility advantage does not seem to compensate for (small) loss in SNR improvement

o Parameters can be tuned to achieve better overal localization performance at the cost of some noise reduction

ConclusionsConclusions

Problem statement





Audio demo

Conclusions

3131

AcknowledgmentsAcknowledgments

Physical and perceptual evaluation of the Interaural Wiener Filter algorithm Simon Doclo 1, Thomas J. Klasen 1, Tim van den Bogaert 2, Marc Moonen 1,

Documents

binaural system

binaural mask

binaural algorithm

preserved noise cues

speech component

htmloverviewbinau hearing

position of speech source

useful speech signalmany