OpenWebRTC – not just for video chat

OpenWebRTC – Not just for Video Chat

Kristofer Dovstam

Master Researcher

Multimedia Applications and Protocols, Ericsson Research

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 2

› Mobile first WebRTC client framework

› Permissive Open Source BSD 2-

clause license

› Independent, interoperable

implementation

› Being integrated into WebKit

› Native applications

› GitHub

What isOpenWebRTC?

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 3

› What is it?

– Not just audio and video, but data too

– Can control reliability and ordering properties depending on needs

– Encrypted real-time data

› OpenWebRTC now with Data Channel support!

– Using the reference SCTP implementation - libusrsctp

– Contributed by Collabora Ltd.

– Has been tested in real use cases…

WebRTC Data Channel

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 4

› Remote Control over Mobile Networks

– Remote control an excavator from a long

way away

– Use WebRTC technology

– Optimize solution for high-performance and

low-latency

› Demo at MWC 2014

– ‘Toy’ excavator

– Earlier internal version of OpenWebRTC

– iPhone

– Oculus Rift DK1

IDEA and History

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 5

…2015

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 6

Excavator

› Research collaboration

with Volvo CE

› Not a product!

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 7

Excavator

› Excavator

– Excavators from Volvo CE

– Modified to handle electrical control

signals

– Powerful Linux PC

– Python application with custom

signaling built on top of

OpenWebRTC

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 8

Audio / VideoCapture

› 2048x2048 24fps

camera

› 360 degree lens

› 52.5 degrees up, 62.5

degrees down

› Avoids stitching artifacts

› Microphone array of four

microphones

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 9

Control Rig› Simulator from Oryx and

Volvo CE

› Mac OS X computer

› OS X Cocoa application

with custom signaling

built on top of

OpenWebRTC

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 10

› Oculus Rift DK2 (1920x1080 75Hz –

960x1080 for each eye)

– Still much too low resolution!

› Doughnut to panorama texture

mapped on a sphere

› Local rendering minimizes motion-to-

light latency of head movements

› Four channel audio rendered as four

independent sound sources in a 3D

space

Audio / VideoRendering

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 11

Controls

› Simulator rig with analogue controls

› Computer measures analogue signals

and converts to digital measurements

› Control signals sent via WebRTC Data

Channel to excavator

› Excavator computer converts digital

signals into analogue electrical signals

› It’s alive!

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 12

Media Processing Architecture

Camera

Microphone Array

Excavator

OpenWebRTC

Audio / Video

Data Channel

OpenWebRTC

Audio / Video

Data Channel

Oculus Rift DK2

Headphones

Control Rig

Network

Mobile, fixed, …

Network

Mobile, fixed, …

Audio

Video

Data

Audio

Video

Data

Audio

Video

Data

Audio

Video

Data

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 13

› Data Channel

– 1 stream

– Unordered

– Unreliable (full control signal

state in every message so

only the newest message is

relevant)

– libusrsctp

› Video

– 2048 x 2048

– ~24fps

– H.264

– OpenH264

– 10Mbps

› Audio

– Single 4-channel stream

– Signed 16-bit

– 48kHz

– Opus

– libopus

– 256 kbps total bitrate

Media Stream Specifications

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 14

Mobile world congress - 2015

ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 15