Global Varnish Cluster with GeoDNS

Improving Site Response TimePart 2: Global Varnish Cluster with GeoDNS

1 18.2.2012Kim Stefan Lindholm

SETUP

Same components as before with following additions/modifications:

• Varnish Server: Amazon EC2 Micro Instance (613 MB)

• Varnish Monitoring: New Relic, Munin, RevealCloud

• Edge Locations: Tokyo, California, Ireland

• GeoDNS: EdgeDirector

DNS and newly added Google Analytics + New Relic code may slightly affect load times

2

Front-end optimization can speed up a website but doesn’t tackle network

latency. Content must be brought closer to visitors.

3

LOAD TIME, BEFORE

4

5 s

9 s

14 s

18 s

Los Angeles São Paulo Stockholm Shanghai Sydney

✘ ✘✘ ✘ ✘

Target Speed(1.5 Mbps connection)

Average: <3.5sWorst Case: <5s

AverageWorst Case

Solution: Set up remote Varnish caches and direct visitors to the nearest one

with GeoDNS

5

6

Varnish

Varnish

Incapsula

Website

Varnish

Varnish serves files that it has previously cached (in its assigned geographic area)

Incapsula fetches uncached content from origin server(s)If edge server is unresponsive, DNS failover redirects to Incapsula or another Varnish

Varnish fetches uncached or expired content through Incapsula, never directly from website

7

LOCAL VS. REMOTE VARNISH

• Why would we place a Varnish box in the same region as the website?

• Several cloud application platforms (CloudControl, Heroku, etc.) have Varnish running locally but only default configuration is supported

• Incapsula sets cookies but Varnish doesn't serve cached results to sessions with cookies ➔ limited benefits with default settings

• We’d like to see cache hit rates of up to 99.9 % so by manually configuring Varnish we ensure that it actually does its job

LOAD TIME, AFTER

8

5 s

9 s

14 s

18 s

Los Angeles São Paulo Stockholm Shanghai Sydney

✔ ✔✔ ✔ ✔

AverageWorst Case

Dotted line denotes load time before

Target Speed(1.5 Mbps connection)

Average: <3.5sWorst Case: <5s

9

http://royal.pingdom.com/2010/11/12/real-connection-speeds-for-internet-users-across-the-world/

“The average connection speed for Internet users worldwide, not just among these 50 countries, but all countries, is an average of 1.8 Mbit/s”

We’ll use 1.5 Mbit/s and 5.0 Mbit/s connections in tests that follow

TARGET SPEED, AVERAGE

10

1.5 Mbps <3.5s

<2.5s5.0 Mbps

11

AVERAGE, BEFORE

DNS Provider: DNS Made Easy5 measurements, >20 min intervals

4.0 s

2.5 s

2.7 s

4.9 s

5.8 s

4.4 s

3.5 s

2.9 s

3.5 s

3.8 s

3.7 s

4.9 s6.0 s

<3.5s <3.5s <3.5s

1.5 Mbps

CloudFront

12

AVERAGE, BEFORE

DNS Provider: DNS Made Easy5 measurements, >20 min intervals

1.5 Mbps

Akamai

3.6 s

2.1 s

2.7 s

4.0 s

4.8 s

5.0 s

3.7 s

3.1 s

3.3 s

4.5 s

4.9 s

4.8 s

<3.5s<3.5s<3.5s

6.2 s

13

CDN COMPARISON

DNS Provider: DNS Made Easy5 measurements, >20 min intervals

1.5 Mbps

Akamai

CloudFront

We’ll be using CloudFront for subsequent tests

- 0.4 s

- 0.4 s

0.0 s

- 0.9 s

- 1.0 s

- 0.6 s

- 0.2 s

- 0.2 s

- 0.2 s

- 0.7 s

- 1.2 s

- 0.1 s- 0.2 s

14

AVERAGE, AFTER

DNS Provider: EdgeDirector5 measurements, >20 min intervals

1.8 s

1.9 s

1.8 s

2.8 s

2.5 s

3.0 s

2.1 s

2.3 s

3.4 s

1.5 s

2.6 s

3.1 s2.8 s

<3.5s <3.5s <3.5s

1.5 Mbps

CloudFront

15

AVERAGE, AFTER

DNS Provider: EdgeDirector5 measurements, >20 min intervals

1.2 s

1.5 s

1.1 s

2.1 s

2.0 s

2.4 s

1.7 s

1.8 s

2.9 s

2.0 s

1.9 s

2.6 s2.2 s

<2.5s <2.5s <2.5s

5.0 Mbps

CloudFront

16

TARGET SPEED, WORST CASE

1.5 Mbps <5.0s

<4.0s5.0 Mbps

17

WORST CASE, BEFORE

DNS Provider: DNS Made Easy5 measurements, >20 min intervals

1.5 Mbps

CloudFront

4.4 s

3.2 s

3.2 s

10.4 s

9.8 s

6.5 s

4.3 s

3.1 s

4.2 s

6.0 s

4.5 s

5.3 s

<5.0s <5.0s <5.0s

8.0 s

18

WORST CASE, AFTER

DNS Provider: EdgeDirector5 measurements, >20 min intervals

1.9 s

2.2 s

2.1 s

3.9 s

2.8 s

4.6 s

2.7 s

2.3 s

4.0 s

1.6 s

3.2 s

3.6 s3.6 s

<5.0s <5.0s <5.0s

1.5 Mbps

CloudFront

19

WORST CASE, AFTER

DNS Provider: EdgeDirector5 measurements, >20 min intervals

1.4 s

1.7 s

1.2 s

2.2 s

2.2 s

3.2 s

2.2 s

1.9 s

3.0 s

3.9 s

2.2 s

2.9 s2.8 s

<4.0s <4.0s <4.0s

5.0 Mbps

CloudFront

CDN doesn’t guarantee optimal delivery locations. Where were assets

actually served from?

20

21

AKAMAI LOCATIONS

CloudFront seems to load all files from Seattle which is just a simple geo-IP lookup issue

Files were loaded from 2 continents

How much does a minimal Varnish cluster cost?

22

SERVERS

23

VPS / Cloud Server Locations RAM Storage Price/month

Linode 512 US, UK, Japan 512 MB 20 GB $19.95

Hetzner VQ 7 Germany 512 MB 20 GB €7.90

Zerigo Linux Denver, CO 512 MB 24 GB $19

Amazon EC2 micro US, EU, Asia, Brazil 613 MB 20 GB (EBS) $16.7 / $20.7

NB 1: Varnish should run on a 64-bit serverNB 2: Amazon prices are for California and Tokyo, reserved, heavy utilization, 30 GB traffic

GEODNS

24

Zerigo DNSPro 4

DNS Made EasyBusiness + GTD

DNS.comBasic / Pro

EdgeDirector+ Monitoring

Monthly Queries 4 million 10 million 5 / 15 million Unlimited

Monthly Price $20 $60 $34.95 $1.5 / monitor + $15 / M queries

+ DNS Failover X Included (3) X Included (1)

+ DNSSEC X X Included X

+ Vanity Name Servers

Included Included $44.95 X

NB: For a small website, EdgeDirector can cost less than $3/month

Without DNS Failover2 x Linode server + Zerigo DNS (1 M queries)= 720 USD (~550 EUR) per year

With DNS Failover2 x Linode server + EdgeDirector (1 M queries)= 690 USD (~530 EUR) per year

25 Rates in February 2012

26

LEARNINGS

• Varnish made a real improvement in response time (up to 7 sec) and as it now handles more than 99 % of requests, a website with few DB write operations performs fine even on shared cloud hosting

• Varnish boxes don’t scale up automatically but as DNS failover bypasses failed edge servers, this doesn’t seem like a problem

• GeoDNS isn’t always necessary for accelerating a website: even one Varnish serving all requests globally will significantly improve performance

27

LEARNINGS

• Amazon micro instance isn’t suitable for production use due to high CPU steal

• Setting up Varnish is pretty straightforward, see Part 3 for an example configuration on Amazon EC2

RESOURCES• Cloud & Virtual Private Servers (VPS)

• Linode, http://www.linode.com/

• Hetzner, http://www.hetzner.de/hosting/produktmatrix_vserver/vserver-produktmatrix

• Zerigo, http://www.zerigo.com/vps-servers

• Amazon EC2, http://aws.amazon.com/ec2/

• GeoDNS / Global Server Load Balancing (GSLB)

• http://www.zerigo.com/docs/managed-dns/setting_up_geodns

• http://help.dnsmadeeasy.com/gtd/configure-gtd-with-dns-failover/

• http://dns.com/Help/#configure

• http://edgedirector.com/how/both.htm

28