Nitty Gritty of Adaptive Video Transmuxing in JS

Adaptive Video Transmuxing

Donato Borrello@CodeAssault

Lead Engineer, JW Player

The Nitty Gritty of

In JavaScript

Take MPEG-TS and make it MP4

What’s goin’ on wit client-side video

Like this ------------>

A 10,000 foot view

What’s a Provider?

Why HTTP Live Streaming?

HLS is the most widely used ABR streaming format

Research

Take it? or Make it?

Opportunities for improvement

➔ Faster time to first frame

◆ Incremental transmuxing instead of batch

◆ “Jump-start” downloads

◆ Clever Parsing using FSM

➔ More feature-rich

◆ Multi-track audio (HLSv4)

➔ More streams supported

◆ A frustratingly thorough test suite

Ultimately, we can take the best parts from each

Caterpillar is born!

Caterpillar Shaped Architecture!

Adaptive Video Streaming over HTTP

Transmuxing with Incremental Computation

Transmuxing Engine

Events or FRP

➔ RxJS using Observable objects

➔ Event based stream philosophy (push streams)

◆ Equates to batch processing

➔ We want Pull Streams

◆ Backpressure with buffers

➔ Quite complex - unused features and hard to debug

➔ Profiling was challenging - trampoline what?

Communicating Sequential Processes

➔ Enter CSP

◆ Channels, processes and buffers

◆ Using generators

➔ Does exactly as much work as needed

➔ Cleaner code

◆ Easier profiling and debugging

Process ChannelChannel

“Jump-start” Downloading

A 10,793 Line Media Manifest

248 kb file size

At 1 Mbps, that’s 250ms of

added latency until download

completes

Manifest Parsing

➔ There is a big async dependency

More typical examples

DVR Streams - worst offenders

Live Streams - rarely a problem

VOD Streams - longer movies see

improvement

The media playlist for Captain

America - Winter Soldier is

1500 lines, 27kb

NALU Parsing with Finite State Machines

PES > NALU > H264 > MP4

That extra step for video PES Packets

I Started Dreaming in Hex

TS Packet Starts

0x47

PES Packet

Starts

NALU starts

0x000001

Finding NAL Units

They don’t line up with anything.

They can start and end anywhere.

All we know is 0x001 or 0x0001

Miscellanea

Adaptive Switching

➔ Time to download

➔ Time to Transmux

➔ Network Volatility

➔ Manifest accuracy

➔ Available Qualities

➔ Individual stream status

➔ Generators screwed us (blame v8)

◆ Babel-Regenerator saved us

➔ UglifyJS is only partially compatible with ES6

➔ Webpack + Babel + Typescript was a hassle

Impact of using ES6

Thanks!

@JWDevelopers

Rob Walch

Kyle Aurand

Jong Park

Evol Greaves

Danny Finkelstein

Mohammed Siddique

Sid Patel

Eric Boyd

Andrew Prelusky

Pooja Maden

@Google

Joey Parrish

@Chromium

Chris Cunningham

Dale Curtis

@gliese1337/HLS.js

Logan Kearsley