Scaling a Web Service

Scaling a Web ServiceBarCamp HK 2007

Leon Ho, Stepcase Limited

Objective

• Sustaining the scale of the traffic

• Zero downtime

• High performance

• Squeezing every juice from the server

Why?

• Stepcase Lifehack (http://www.lifehack.org):

• 1 million visits per month

• 2 million pageviews per month

• Those does not include images, javascript, stylesheets

• We’ve done 5400 new connections per min

1. Hardware

Shared Hosting

Web Server DB

Stage 1

Web Server DB

Stage 2

Web ServerMaster

DB

SlaveDB

SlaveDB

Stage 3

MySQL Cluster

Web Server

MasterDB

MasterDB

Stage 4

MySQL Cluster

App ServerMaster

DB

MasterDB

App Server

App Server

Load Balancer

Stage 5

Servers

• Stage 2

• Running 2 servers; each with 4 cores and 4GB RAM

• Exposing only one server to public network

Remember the point is to stay in early stages.

2. Reduce Connections

Static Files

• Moves out to a separate server

• Consider Amazon S3

• Image - Decrease the number of images you need to load

• CSS Spites

• Use Firebug to benchmark

Amazon S3

• We moved our static images to S3

• files.lifehack.org CNAME s3.amazonaws.com

• use cronjob and s3sync to keep them in sync

• next, move js, css, and post images out

Feed

• 60,000 subscriptions daily

• We use external provider - Feedburner

3. Reduce File Size

Javascript

• Compress the javascript

• JSMin, Dojo ShrinkSafe, Packer, YUI Compressor

• http://compressorrater.thruhere.net

4. Optimize and Cache

Web & DB Server

• Web: Moved to Lighttpd

• DB: MySQL. Increase query cache, max connection and wait timeout

Cache

• Cache, Cache, and Cache

• Page level (wp-cache)

• DB queries (memcache, or any object caches)

• PHP opcode - xcache, e-accelerator

What else?

• Eliminate bottleneck

• Hard drive > Memory > CPU

• Reduce Abuse and Spam

Next

• Adding more comprehensive monitoring

• Munin

• More scaling - this time - on Rails

Questions?

Visit Us: http://www.stepcase.comhttp://www.lifehack.org

P.S. We are hiring!

Contact Me:[email protected]