А. Патраков "PulseAudio и бытовая электроника: цифровая обработка звука", DUMP-2014

Sound Processingin Consumer Electronics

and in PulseAudio

Alexander Patrakov

About Myself

● Author of some sound processing code– Dcaenc: a Free DTS-compatible encoder

– Wine resampler

● Dayjob:

Hardware

LG 47LM640T

Onkyo TX-NR626

Common effects

● Equalizer presets– Sound Mode, Listening Modes

● Virtual Surround– Theater-Dimensional Mode

● Dynamic Range Compression– Late-Night Mode

● User EQ● ClearVoice II

● Bass, Tremble● Speaker Distance

Compensation● Bass Management● Digital room

Correction

5.1 Speaker SetupDelay is needed!

Instead of the equalizer

It Just Works!

PulseAudio

DSP-related functionality

● Equalizer● Resampler● Channel Remixer● Virtual Surround

● Bass Management● Encoders & Decoders● LADSPA Effects● Echo Cancellation

In this talk

● Equalizer● Resampler● Channel Remixer● Virtual Surround

● Bass Management● Encoders & Decoders● LADSPA Effects● Echo Cancellation

module-equalizer-sink

Author: Jason Newton

Spectrum * Filter

Spectrum * Filter

?

Convolution

t

FIR filter design

● Given the desired frequency response– Take IFFT

– Shift the signal to positive time values

– Truncate the signal to a suitable length

– Apply a windowing function

http://www.dspguide.com/ch17/1.htm

Divide and Conquer

FFT-based magic

Circular convolution

t

Convolution

t

Tricks Against Circular Convolution

● Overlap-Save● Overlap-Add

Overlap-Save

Corrupted fragment

The input signal is split into overlapping segmentsCorrupted fragments are thrown away(the previous segment contains a non-corrupted version)

Overlap-Add

Space for growth

Pad the input signal fragments with zeroesPerform circular convolutionResult: overlapped segments of output signalResolve the overlap by summation

module-equalizer-sink

● Where is this?– Take IFFT

– Shift the signal to positive time values

– Truncate the signal to a suitable length

– Apply a windowing function

module-equalizer-sink

?

Latency = ?

module-equalizer-sink

/* FIXME: Please clean this up. I see more commented code lines * than uncommented code lines. I am sorry, but I am too dumb to * understand this. */

fftwf_malloc(); …; free();

Buffer Overflow

module-equalizer-sink

EPIC FAIL

module-ladspa-sink + mbeq + veromix

module-virtual-surround-sink

5.1 recording Stereo Headphones

Author: Niels Ole Salscheider

Amplitude difference

Woof!Woof!

Delay

Woof..!...Woof!

Spectral features

module-virtual-surround-sink

● Convolution with HRIR (Head Related Impulse Response)– HRIR has to be downloaded separately

Virtual Surround on Speakers

● Ambiophonics– Speaker crosstalk cancellation

– Same technology as in soundbars

– Not in PulseAudio yet

HRIR (left ear)

HRIR (left ear)

Cutoff at 64 samples(to save CPU time)

HRTF (left ear)

HRTF (left ear)

This sounds cheap

Conclusions

Open Tasks

● See GSOC thread: http://goo.gl/7fVwTk

Other Relevant Projects

● Speex: contains a denoiser● Vlevel: dynamic range compressor● BruteFIR: a fast convolution engine● DRC-FIR: digital room correction