Y!7 Simple Scalability

Simple Scalability

On the cheap.

Who am I?

Architect at Yahoo

Policy Advisor

Lamp Developer

Welfare Advocate

The Basics

scale vs ha

horizontal vs vertical

write vs read

storage

serving

database

Scale vs HA

Scale is NOT– Redundant– Performance*– Durable

HA is NOT– Distributed– Performance– Easy

http://www.studiosysadmins.com/gallery/detail/31/

Horizontal vs Vertical

Vertical– Throw money at the problem– Disk, CPU, Ram.– Generally, good start.

Horizontal– Throw smarts at the problem– Strategic endpoints– Smarter applications

Scale, how?

Distribute the storage– Shared Disk, NFS, Etc.

Distribute the db– Cluster, read nodes, etc.

Distribute the application– End points, read points, write points

Write vs Read

Separate application logic

DB Abstraction is a good place to start– Run compare against query to detect method and select db profile for action

Disk write points– Pick a master server (simple) and sync out from there

Basic Site

User access website

Website uses DB

Website stored on Disk

Some static content on Disk

Website

Dynamic ContentStatic Content

Typical Scenario

Read Mostly

Write Rarely (Database)

Uploads (Storage)

More DB Content than static

Some static content (images, CSS, etc)

Dynamic ContentStatic Content

Storage

Use a central location

Same Rack/Location– NFS, iSCSI, other solutions

Low write, mostly read– Scheduled syndication– Master location

Globally positioned– CDN for statics, API for write, DB read

Manage detection, syndication, failure

Serving

Syndicating Application Files– Bit Torrent– RSync– CVS/SVN (Versioning in general)– Complex scripts

Managing users– Sessions!

Authority, Security

Shared nothing (SNA)

Database

Cluster– Good, Bad, Indifferent?

Read Nodes vs Write Nodes

Amazon RDB

Multi Master, Master Master, Master Slave

Don’t touch HA

Accessing Scaled Data

API

Read, Write Points

Data Exchange

Databases

Managing Scaled Data

Versioning

Failures

Syndication of ‘stuff’

Authority– Master servers are always great

http://hackedgadgets.com/2006/02/04/server-farm/

Geo DNS

Like DNS Load Balancing

Geared to IP pools, vs actual geography

Newish, not overly supported

Great for distributing traffic to multiple LB or etc.

Australian Host USA Host

AU US

Geo DNS

Tag as many countries to specific services as you like

Allow for ungrouped countries to go somewhere different

Full CName, A, AAAA support

Australian Host USA Host

USEU

SG

CANZAU

Scale on a Budget

Amazon Web Services– S3, EC2, VPC, RDS, SNS

Zerigo – Cloud DNS, Geo DNS, VPS

Mammoth – VPS

Questions

Ask me later

@steveathon

http://fwdpass.it/bN4qeh