MBA - 2015 Working in open collaboration, to build a transparent measurement collection and reporting methodology, and to publish a freely available dataset.

MBA - 2015Working in open collaboration, to build a transparent measurement collection and

reporting methodology, and to publish a freely available dataset of US consumer broadband

performance, since 2010.8 April 2015 / 18 August 2015

AGENDA•Overview•Project Objective•Selected Services (beta)•Timeline•Review of Technical Methodology• Joint Opportunities•UCSC initial results

Project Objective

Create a working group whose purpose is to develop a common methodology of measuring video streaming performance to allow for a comparative measurement and analysis of available services.

Review of Technical Methodology

Whitebox

Network BEFORE whitebox

Network with whitebox

Whitebox operation•Whitebox defers scheduled tests when the home

network is active•Wired traffic is detected passing through the whitebox•Wi-Fi radio is used to detect network activity from

wireless clients•Whitebox does not need your key to detect activity•Whitebox reports total GB/month traffic• FCC interest in actual v advertised bandwidth

YouTube Methodology•Streams real videos from YouTube•Uses the real YouTube servers (including Google Global Cache, if installed by the ISP•Tests most popular videos•Decodes videos on the probes – it’s not just a speed test

Netflix Methodology• Streams synthetic content from Netflox•Uses the real Netflix ‘Open Connect Appliances’ inside

the ISP network•Downloads content at full-rate for 20 seconds. A

bitrate reliably streamed’ metric can be derived from analysis of this transfer•Developed in collaboration with Netflix•Complaints about use of synthetic data

Beta Observations (April 2015)

• Early rollout showed peak-hour congestion issues in certain areas for both YouTube and Netflix•Many tests operate over IPv6 rather than IPv4, again

for both YouTube and Netflix• Tests new report the maximum available bitrate of the

video as well, so it is possible to present the bitrate reliably streamed as a percentage.

UCSC test plan

•UCSC has a 200 node DOCSIS 2 cable modem service•Built in 1970, before the Internet, before dirt• This cable TV coax was not designed for data•We will compare DOCSIS modem service to campus

Ethernet•One test box in Engineering building•One test box in a Grad Student Appt

Netflix StreamingCENIC has Netflix caches in LA and Sacramento

More info

• The FCC uses contractor SamKnows to code and deploy measurement infrastructure• https://www.samknows.com/• https://www.samknows.com/broadband/uploads/methodology/SQ30

1-005-EN-Test-Suite-Whitepaper-4.pdf• They are still sending out whiteboxes to candidate participants• Testing of a 1 Gb/s whitebox has started• FCC’s annual broadband report is late, but soon – they say.