Python, WebRTC and You (v2)

Python, WebRTC and YouSaúl Ibarra Corretgé

@saghul

v2

github.com/saghul

WebRTC, anyone?

Have you ever used it?

Internals, anyone?

What is WebRTC?

WebRTC (Web Real-Time Communication) is an API definition drafted by the World Wide Web Consortium (W3C) that supports browser-to-browser applications for voice calling, video chat, and P2P file sharing without the need of either internal or external plugins.

Well, everyone better Restart Their Chrome

You need an adapter

Implementation in browsers is currently inconsistent

Some APIs are still in flux

rtcninja.js

https://github.com/eface2face/rtcninja.js

Nice name, right?!

Temasys WebRTC Plugin

Free (as in beer) plugin for IE and Safari

http://skylink.io/plugin/

WebRTC APIs

getUserMedia

RTCPeerConnection

RTCDataChannel

getUserMediaif (!rtcninja.hasWebRTC()) { console.log('Are you from the past?!'); return;}!rtcninja.getUserMedia( // constraints {video: true, audio: true},! // successCallback function(localMediaStream) { var video = document.querySelector('video'); rtcninja.attachMediaStream(video, localMediaStream); },! // errorCallback function(err) { console.log("The following error occured: " + err); });

RTCPeerConnection

Handles streaming of media between 2 peers

Uses state of the art technology

JSEP

RTCPeerConnection (2)Get local media Send SDP offer

Get local media

Send SDP answer

ICE candidates

Audio / Video

Interactive Connectivity Establishment

ICEHelps find the best path for media

Solves NAT traversal and other hostile network problems

Communication Consent Verification

It can trickle!

What about the signalling?

It’s not specified!

Use SIP, XMPP or roll your own!

RTCDataChannel

P2P, message boundary based channel for arbitrary data

Implemented using SCTP, different reliability choices possible

Call Roulette

Python

JavaScript

The ProtocolWebSocket based, JSON payload

Users enter the roulette when they connect over WebSocket

Session is negotiated / established

No end message, just disconnect the WebSocket

Saghul’s Imbecile Protocol (v1)

yo (v2)

{'yo': 'yo'}

{'jsep': {'sdp': '...', 'type': 'offer'}, 'yo': 'yo'}

{'jsep': {'sdp': '...', 'type': 'answer'}, 'yo': 'yo'}

{'candidate': {'candidate': '...', 'sdpMLineIndex': 1, 'sdpMid': ''}, 'yo': 'yo'}

Shopping for a framework

Python >= 3.3, because future!

WebSocket support built-in

Async, because blocking is so 2001

New, because why not?

asyncio + aiohttp

github.com/saghul/CallRoulette

@asyncio.coroutinedef init(loop): app = web.Application(loop=loop) app.router.add_route('GET', '/', LazyFileHandler(INDEX_FILE, 'text/html')) app.router.add_route('GET', '/ws', WebSocketHandler()) app.router.add_route('GET', '/static/{path:.*}', StaticFilesHandler(STATIC_FILES))! handler = app.make_handler() server = yield from loop.create_server(handler, '0.0.0.0', 8080) print("Server started at http://0.0.0.0:8080") return server, handler

class StaticFilesHandler: def __init__(self, base_path): self.base_path = base_path self.cache = {}! @asyncio.coroutine def __call__(self, request): path = request.match_info['path'] try: data, content_type = self.cache[path] except KeyError: full_path = os.path.join(self.base_path, path) try: with open(full_path, 'rb') as f: content_type, encoding = mimetypes.guess_type(full_path, strict=False) data = f.read() except IOError: log.warning('Could not open %s file' % path) raise web.HTTPNotFound() self.cache[path] = data, content_type log.debug('Loaded file %s (%s)' % (path, content_type)) return web.Response(body=data, content_type=content_type)

class WebSocketHandler: def __init__(self): self.waiter = None! @asyncio.coroutine def __call__(self, request): ws = web.WebSocketResponse(protocols=('callroulette-v2',)) ws.start(request)! conn = Connection(ws) if self.waiter is None: self.waiter = asyncio.Future(loop=ws._loop) fs = [conn.read(), self.waiter] done, pending = yield from asyncio.wait(fs, return_when=asyncio.FIRST_COMPLETED) if self.waiter not in done: # the connection was most likely closed self.waiter = None return ws other = self.waiter.result() self.waiter = None reading_task = pending.pop() reading_task.cancel() asyncio.async(self.run_roulette(conn, other)) else: self.waiter.set_result(conn)! yield from conn.wait_closed()! return ws

from jsonmodels import models, fieldsfrom jsonmodels.errors import ValidationError!!class StringChoiceField(fields.StringField): def __init__(self, choices=None, *args, **kw): self.choices = choices or [] super(StringChoiceField, self).__init__(*args, **kw)! def validate(self, value): if value not in self.choices: raise ValidationError('invalid choice value') super(StringChoiceField, self).validate(value)!class Jsep(models.Base): type = StringChoiceField(choices=['offer', 'answer'], required=True) sdp = fields.StringField(required=True)!class Candidate(models.Base): candidate = fields.StringField(required=True) sdpMid = fields.StringField(required=True) sdpMLineIndex = fields.IntField(required=True)!class YoPayload(models.Base): yo = fields.StringField(required=True) jsep = fields.EmbeddedField(Jsep) candidate = fields.EmbeddedField(Candidate)

@asyncio.coroutine def run_roulette(self, peerA, peerB): log.info('Running roulette: %s, %s' % (peerA, peerB))! @asyncio.coroutine def close_connections(): yield from asyncio.wait([peerA.close(), peerB.close()], return_when=asyncio.ALL_COMPLETED)! def parse(data): try: data = json.loads(data) payload = YoPayload(**data) payload.validate() except Exception as e: log.warning('Error parsing payload: %s' % e) return None return payload

# request offer offer_request = YoPayload(yo='yo') peerA.write(json.dumps(offer_request.to_struct()))! # get offer data = yield from peerA.read(timeout=READ_TIMEOUT) if not data: yield from close_connections() return! offer = parse(data) if offer is None or offer.jsep is None or offer.jsep.type != 'offer': log.warning('Invalid offer received') yield from close_connections() return! # send offer peerB.write(json.dumps(offer.to_struct()))

# wait for answer data = yield from peerB.read(timeout=READ_TIMEOUT) if not data: yield from close_connections() return! answer = parse(data) if answer is None or answer.jsep is None or answer.jsep.type != 'answer': log.warning('Invalid answer received') yield from close_connections() return! # dispatch answer peerA.write(json.dumps(answer.to_struct()))

# wait for candidates / end while True: peer_a_read = asyncio.async(peerA.read()) peer_a_read.other_peer = peerB peer_b_read = asyncio.async(peerB.read()) peer_b_read.other_peer = peerA done, pending = yield from asyncio.wait([peer_a_read, peer_b_read], return_when=asyncio.FIRST_COMPLETED) for task in pending: task.cancel() for task in done: data = task.result() if not data: break # all we can get at this point is trickled ICE candidates candidate = parse(data) if candidate is None or candidate.candidate is None: log.warning('Invalid candidate received!') break task.other_peer.write(json.dumps(candidate.to_struct())) else: continue break # close connections yield from close_connections()

In WebRTC trouble?

bettercallsaghul.com@saghul