Spatial Conferencing

© Fraunhofer IIS 1 Spatial Conferencing

Redmond, 16 Oct 2014

Spatial Conferencing

HP Baumeister

© Fraunhofer IIS 2 Spatial Conferencing

Telepresence Rooms: High End Conferencing

© Fraunhofer IIS 3 Spatial Conferencing

General Reaction: “As if speaking to someone in the same room”

© Fraunhofer IIS 4 Spatial Conferencing

General Reaction: “As if speaking to someone in the same room”

Why?

In part, because the audio is “transparent”, and comes from the appropriate

direction!

“Spatial Audio”

© Fraunhofer IIS 5 Spatial Conferencing

“Spatial Audio”

• audio: 3 channels of Fraunhofer AAC-LD @ 48kHz

• video: 3 x H.264 @ 1080p

© Fraunhofer IIS 6 Spatial Conferencing

Telepresence for Everyone: Spatial Conferencing

Bring the “telepresence experience” to smartphones

Using regular headsets/earphones

© Fraunhofer IIS 7 Spatial Conferencing

Telepresence for Everyone: Spatial Conferencing

“Billions of smartphone and tablets”

© Fraunhofer IIS 8 Spatial Conferencing

Spatial Conferencing: Key Elements

Highest Quality Audio: “Full-HD Voice”

“Spatialization”

© Fraunhofer IIS 9 Spatial Conferencing

Spatial Conferencing: “Full-HD Voice”

14 kHz+ audio bandwidth

© Fraunhofer IIS 10 Spatial Conferencing

Full-HD Voice Audio Quality

source: http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_52/Docs/S4-090080.zip (Feasibility study on EVS audio bandwidth, Ericsson 3GPP/SA4 01/2009)

0 1 2 3 4 5

Mean Opinion Score

Fair Good Excellent Bad Poor

NB

WB

SWB

© Fraunhofer IIS 11 Spatial Conferencing

Spatial Conferencing “Full-HD Voice”

14 kHz+ audio bandwidth

Very low delay

Multi-channel capability

© Fraunhofer IIS 12 Spatial Conferencing

Spatial Conferencing “Full-HD Voice”

ELD = Enhanced Low Delay

AAC-ELD is a low latency mode of the widely adopted music codec AAC.

CD-like audio quality for telephone calls

full audio bandwidth up to 20 kHz

low coding delay down to 15 ms, crucial for natural conversations

optimized for a bit-rate range of 24 kbit/s to 64 kbit/s per channel

AAC-ELDv2 enhances bit-rate range down to 24 kbit/s stereo

Most widely adopted codec offering Full-HD Voice today

FaceTime

Native in iOS

Native in Android

© Fraunhofer IIS 13 Spatial Conferencing

Spatial Conferencing for Everyone “Spatialization”

Geometric Localization of Participants

With or without Video……

© Fraunhofer IIS 14 Spatial Conferencing

Spatial Conferencing “Head Related Transfer Function”

© Fraunhofer IIS 15 Spatial Conferencing

Spatial Conferencing “Head Related Transfer Function”

Credit: University of Maryland

© Fraunhofer IIS 16 Spatial Conferencing

Spatial Conferencing “Out of Head Experience”

Stereo = “Inside Head”

Spatial Rendering = “outside Head”

© Fraunhofer IIS 17 Spatial Conferencing

Audio Conferencing

MCU

© Fraunhofer IIS 18 Spatial Conferencing

Audio Conferencing “single channel” – “everyone is in the same location”

MCU

© Fraunhofer IIS 19 Spatial Conferencing

Spatial Conferencing “Spatialization” – “participants can be localized as if in the room”

MCU

positioning &

out of head

Full-HD Voice (spatialized)

© Fraunhofer IIS 20 Spatial Conferencing

Spatial Conferencing MCU

© Fraunhofer IIS 21 Spatial Conferencing

3-Point conference via IP

Full Band Audio, 44.1 kHz

Immersive, spatial experience

Spatial Conferencing Demo Setup

3x Nexus 5 - Android 4.4.3.2.1 - VoIP App using native ELD

standard in-ear headset

Mac-Mini running MCU with binaural processing

64 kbps mono

128 kbps stereo

© Fraunhofer IIS 22 Spatial Conferencing

Developed and evaluated in collaboration with

Spatial Conferencing Trials

Full-HD VoicePSTN

“I found it really amazing, because one really

thought you are sitting in the same room”

© Fraunhofer IIS 23 Spatial Conferencing

Member Companies should consider spatial conferencing in future systems

Define uses cases

Interface Specification / Standardisation

Interoperability

Spatial Conferencing IMTC Opportunities

© Fraunhofer IIS 24 Spatial Conferencing

See me: HP Baumeister

Call me: 408 573 9903

Mail Me [email protected]

Thank you! Questions? Live Demo?

© Fraunhofer IIS 25 Spatial Conferencing

Spatial Conferencing “Spatialization”

Full-HD Voice (mono)

MCU

positioning &

out of head

Full-HD Voice (spatialized)