Big Internet Techniques Josh Gagliardi – CTO, Highwinds UCF COT 4210 12 Apr 2012

Big Internet Techniques

Josh Gagliardi – CTO, HighwindsUCF COT 4210

12 Apr 2012

Company ProfileOverview• Global provider of CDN, IP Software and Content

Storage• $100M business, highly profitable and financially stable • High-capacity backbone with >3 terabits per second of

extensively peered network capacity and PoPs on three continents

• Offices: Winter Park, FL; Phoenix, AZ; Amsterdam, Netherlands; Costa Mesa, CA; São Paulo, Brasil

• 175 employees

Network Features• >3.5 Tb/s total capacity to the Internet• >650 Gb/s sustained traffic• 56 POPs on 3 continents• 12,000+ BGP peering sessions with end-user networks • ~32 PB of storage across 3,000 servers (<2% NAS/SAN)

Code/OS Statistics• ~2.5 million lines = ~1 million production, ~1.5 million

test• C++, Java, Perl, PHP, Flex, bash• 85% server, 10% client, 5% glue• 95% non-DB storage

Bandwidth

Mind-Numbing Complexity

Simplified for Customers

Simplified for Customers

Problem Decomposition

1. Designing2. Bootstrapping3. Scaling

Design

• WHAT IS THE ESSENCE OF THE PROBLEM?• Define State• Ω(n) – proxies needed

– count the 1:many mappings– count the IOs

Bootstrap

• Other People’s Code• Other People’s Gear• Generic Solutions

– DB, SAN, Apache, PHP• Launch early / often

Scale

• This is where fortunes are made.• Internet -> quick success, quicker failure• Statistics

– Statistical Process Control– Tricks -> { ∑x , ∑x2, N } -> stdev; R; Processing.org

• Systems• Hashsplit / Random• Approximate / Fixed-Space Algorithms

US and Europe Results

The Gear Problem

The Lifetime Problem

• Rate of Technology Change• Snake Oil ( HA, SSD, cloud computing )• 1996 – 2012 programmer view

– IDE -> EIDE -> ATA -> SATA– SCSI -> iSCSI/FCAL -> SSD

• Peeing in the Pool– Microsoft / HTTP / firewalls / the Location header– Criminals / Fraudsters / Script Kiddies

Fixed-Space Algorithms

• Router Guys – George Varghese• Big Data – {ORCL++, EMC++}• Algorithmic Techniques

1. Probabilistic Counting2. Bloom Filters3. Parallel Distributed Filters4. Ratkiewicz Trending

• Perspective Shift– Long Tail / The “Other” bucket

The “Other” Bucket

Josh GagliardiCTO, Highwinds+1.407.215.2432

[email protected]

About Highwinds:www.highwinds.com

Programming Internships Available