WebRTC + Socket.io: building a skype-like video chat with native javascript

WEBRTC + SOCKET.IOBUILDING A SKYPE-LIKE VIDEO CHAT WITH

NATIVE JAVASCRIPT

/ MICHELE DI SALVATORE @MIKDISALJavascript Architect @ Objectway

Audio and video communication and peer-to-peer data

sharing through a web application

Native javascript (no plugins)

Open source

appear.in

HOW IS BORNGlobal IP Solutions acquired by Google in 2010

Google open sourced the technologies

SUPPORTED BROWSERS AND PLATFORMS

IE AND SAFARI? freeTemasys Plugin

JAVASCRIPT WEBRTC APIMediaStream

RTCPeerConnection

RTCDataChannel

MEDIASTREAM (AKA GETUSERMEDIA)Acquiring audio and video

Can contain multiple 'tracks'var constraints = { video: { mandatory: { minWidth: 640, minHeight: 360 } };

function successCallback(stream) { var video = document.querySelector("video"); video.src = window.URL.createObjectURL(stream); }

function errorCallback(error) { console.log("navigator.getUserMedia error: ", error); }

Constraints controls the contents of the MediaStream

RTCPEERCONNECTION

Communicating audio and video

Signal processingCodec handlingPeer to peer communicationSecurityBandwidth management...

RTCDATACHANNEL

Communicating arbitrary data

Same API as WebSocketsUltra-low latencyUnreliable or reliableSecure

SIGNALINGExchanging 'session description' objects

What formats I support, what I want to sendNetwork information for peer-to-peer setup

Any message mechanism and protocol

JSEPJavaScript Session Establishment Protocol

RTCSESSIONDESCRIPTION EXAMPLE

v=0 o=- 7614219274584779017 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS m=audio 1 RTP/SAVPF 111 103 104 0 8 107 106 105 13 126 c=IN IP4 0.0.0.0 a=rtcp:1 IN IP4 0.0.0.0 a=ice-ufrag:W2TGCZw2NZHuwlnf a=ice-pwd:xdQEccP40E+P0L5qTyzDgfmW a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=mid:audio a=rtcp-mux a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:9c1AHz27dZ9xPI91YNfSlI67/EMkjHHIHORiClQea=rtpmap:111 opus/48000/2 ...

P2P IN REAL WORLD

NAT & Firewalls?

STUN SERVERGives my public IP addressSimple server, cheap to runData flows peer-to-peer

STUN SERVER

TURN SERVERFallback for p2p failingData via serverCall works everywhere

STUN + TURN SERVERS

ICEInteractive Connectivity Establishment

A framework to connect peers via UDP, enablingRTCPeerConnection via STUN and TURN server

WEBRTC SECURITYOn Chrome only via https

Encryption for media and data

Sandboxed

WEBRTC LIBRARIES & CO.

by by by

by ...

adapter.js webrtcrtc-everywhere contraSimpleWebRTC &YetRTCMultiConnection Muaz Khan

RTCMULTICONNECTION

A javascript wrapper library supportingapproximately all possible peer-to-peer

features.

WEBRTC EXPERIMENTS

Tons of open source experiments based onRTCMultiConnection:

webrtc-experiment.com

WEBSOCKETS

Standardized in 2011

Interactive communication session between browser andserver

Event driven

Sending strings and binary data

WEBSOCKETS BROWSER SUPPORT

All and IE10+

ajax polling as fallback

SOCKET.IO

A popular open source library with client and serverimplementations

Server side written in Node and easy to startvar server = require('http').createServer(); var io = require('socket.io')(server);

io.on('connection', function(socket){ socket.on('hi!', function(data){ socket.emit('a-message', {hello: 'world'}); });

socket.on('disconnect', function(){}); });

server.listen(3000);

SOCKET.IO CLIENT

var socket = io.connect('http://localhost:3000/');

socket.on('connect', function () { socket.emit('hi!'); });

socket.on('a-message', function (data) { console.log(data.hello); });

GOALA SKYPE LIKE CHAT

MAIN ISSUESWebRTC architecture like IRC

rooms with shared chat: text, video and datadirect chat: only one to oneroom managed by initiatorconversation type can change without agreement: text tovideo

IMPLEMENTATION

STEP 1

UserA is logged in and is connected to the socket server

STEP 2

The app sends a "presence" event to other users throughthe socket

The server forwards the message only to user's contacts

Each present user replies with the same "presence" event

SOCKET.IO FORWARD ROLE

STEP 3

Each present user replies with the same "presence" event

STEP 4

When a user receives a "presence" event, it automaticallyknows that the sender is online

Also the server sends the “presence” event to all, when auser socket is disconnected

STEP 5

UserA starts a conversation and the app creates the webrtcconnection

STEP 6

UserA sends a message via webrtc

STEP 7

UserB receives a notification

STEP 8

UserB starts writing a message

STEP 9

UserA receives the "is typing" info sent via webrtc

STEP 10

UserA starts a video call sending a request to UserB viasocket

STEP 11

UserB receives the request and accepting starts a video callvia webrtc

STEP 12

The 2 users can video chat via webrtc

Each user can close the video at any time or start a text chat

STEP 13

A user can send text messages also

STEP 14

A user can start an audio chat in the same way as a videochat

chat

BEHIND THE SCENES

WHERE IS WEBRTC?

Behind this actions

chat messagesvideo / audio call"is typing" info

AND THE OTHERS?via WebSockets

presencevideo / audio initializationall status infos

THAT'S ALL!THANK YOU ALL

/ Michele Di Salvatore @MikDiSal