Scaling up to your first 10 million users - Amazon S3...Scaling up to your first 10 million users Matthijs ten Seldam Solutions Architect - AWS. ... • DB caching to Amazon ElastiCache

www.techarchday.fiwww.techarchday.fi

Scaling up to your first 10 million users

Matthijs ten Seldam

Solutions Architect - AWS

www.techarchday.fi

So How Do You Scale on AWS?

www.techarchday.fi

www.techarchday.fi

www.techarchday.fi

Auto Scaling

A tool A destination

www.techarchday.fi

What Do We Need First?

www.techarchday.fi

Some Basics…

www.techarchday.fi

US-WEST (Oregon)

EU (Ireland)

ASIA PACIFIC

(Tokyo)

US-WEST (N. California)

SOUTH

AMERICA (Sao

Paulo)

US-EAST (N. Virginia)

AWS GOVCLOUD (US)

ASIA PACIFIC

(Sydney)

ASIA PACIFIC

(Singapore)

CHINA (Beijing)

ASIA PACIFIC

(Seoul)

RegionsEU (Frankfurt)

INDIA (2016)

UK (2016)

Canada

Ohio

Ningxia

www.techarchday.fi

US-WEST (Oregon)

EU (Ireland)

US-WEST (N. California)

SOUTH

AMERICA (Sao

Paulo)

US-EAST (N. Virginia)

AWS GOVCLOUD (US)

ASIA PACIFIC

(Sydney)

ASIA PACIFIC

(Singapore)

Availability ZonesEU (Frankfurt)

INDIA (2016)

UK (2016)

Canada

ASIA PACIFIC

(Tokyo)

CHINA (Beijing)

ASIA PACIFIC

(Seoul)Ohio

Ningxia

www.techarchday.fi

Edge Locations

www.techarchday.fi

ENTERPRISE APPS

Streaming Data Collection

Machine Learning

Elastic Search

Virtual Desktops

Sharing & Collaboration

Corporate Email

Backup

Search

Email

Transcoding

Sync

Single Integrated Console

PushNotifications

Application Lifecycle Management

Containers

Triggers

Resource Templates

RegionsAvailability Zones

Points of Presence

INFRASTRUCTURE

CORE SERVICES

ComputeVMs, Auto-scaling, & Load Balancing

StorageObject, Blocks, Archival, Import/Export

DatabasesRelational, NoSQL, Caching, Migration

NetworkingVPC, DX, DNS

CDN

Access ControlIdentity Management

Key Management & Storage

Monitoring & Logs

Assessment and reporting

Resource & Usage Auditing

SECURITY & COMPLIANCE

Configuration Compliance

Web application firewall

IoT

Rules Engine

Device Shadows

Device SDKs

Registry

Device Gateway

Streaming Data Analysis

MobileAnalytics

The AWS Platform

HYBRIDARCHITECTURE

Data Backups

Integrated App Deployments

DirectConnect

IdentityFederation

IntegratedResource Management

Integrated Networking

MARKETPLACE

Business Apps

Business Intelligence

DatabasesDevOps Tools

NetworkingSecurity Storage

DEVELOPMENT & OPERATIONSMOBILE SERVICESAPP SERVICESANALYTICS

DataWarehousing

Hadoop/Spark

Queuing & Notifications

Workflow

One-click App Deployment

Identity DevOps Resource Management

API Gateway

Business Intelligence

www.techarchday.fi

AWS Building Blocks

Inherently scalable, highly available andfault-tolerant services

Highly available with the right architecture

Amazon CloudFront

Amazon Route 53

Amazon S3

Amazon DynamoDB

Elastic Load Balancing

Amazon EFS

AWS Lambda

Amazon SQS

Amazon SNS

Amazon SES

Amazon SWF

…

Amazon EC2

Amazon EBS

Amazon RDS

Amazon VPC

www.techarchday.fi

So Let’s Start From…

www.techarchday.fi

1 User

• Amazon Route 53 for DNS

• A single Elastic IP• A single Amazon EC2

instance– With full stack on this

single host• Web app• Database• Management• And so on…

Amazon EC2 instance

Elastic IP

UserAmazon Route 53

www.techarchday.fi

“We’re Gonna Need a Bigger Box”

• Simplest approach

• Can now leverage PIOPS

• High I/O instances

• High memory instances

• High CPU instances

• High storage instances

• Easy to change instance sizes

• Will hit an endpoint eventually

c4.8xlarge

m3.2xlarge

t2.micro

X1 (2TB, > 100 vCPU)

www.techarchday.fi

1 User

• We could potentially get to a few hundred to a few thousand depending on application complexity and traffic

• No failover

• No redundancy

• Too many eggs in one basket

EC2 Instance

Elastic IP

UserAmazon Route 53

www.techarchday.fi

Users > 1

First, let’s separate out our single host into more than one.

• Web

• Database Make use of a database

service?

Web Instance

DatabaseInstance

Elastic IP

UserAmazon Route 53

www.techarchday.fi

Self-managed Fully managed

Database server on Amazon EC2

Your choice of database running on

Amazon EC2

Bring Your Own License (BYOL)

Amazon DynamoDB

Managed NoSQL database service using

SSD storage

Seamless scalability Zero administration

Amazon RDS

Microsoft SQL Server OracleMySQL

PostgreSQLMariaDB

Amazon Aurora

BYOL or license Included

Amazon Redshift

Massively parallel, petabyte-scale data warehouse service

Fast, powerful, and easy to scale

Database Options

www.techarchday.fi

Users > 100

First, let’s separate out our single host into more than one:

• Web

• Database Use Amazon RDS to

make your life easier

Web instance

Elastic IP

RDS DB instance

UserAmazon Route 53

www.techarchday.fi

Users > 1000

Next, let’s address our lack of failover and redundancy issues:

• Another web instance– In another Availability

Zone

• RDS Multi-AZ• Elastic Load Balancing

(ELB)

Web Instance

RDS DB InstanceActive (Multi-AZ)

Availability Zone Availability Zone

Web Instance

RDS DB Instance Standby (Multi-AZ)

ELB Balancer

UserAmazon Route 53

www.techarchday.fi

Scaling Options

ve

rtic

ally

horizontally

www.techarchday.fi

Users > 10,000s–100,000s

RDS DB InstanceActive (Multi-AZ)

Availability Zone Availability Zone

RDS DB Instance Standby (Multi-AZ)

ELB Balancer

RDS DB Instance Read Replica

RDS DB Instance Read Replica

RDS DB Instance Read Replica

RDS DB Instance Read Replica

Web Instance

Web Instance

Web Instance

Web Instance

Web Instance

Web Instance

Web Instance

Web Instance

Amazon Route 53User

www.techarchday.fi

What AboutPerformance and Efficiency?

www.techarchday.fi

Lighten the Load

www.techarchday.fi

RDS DB InstanceActive (Multi-AZ)

Availability Zone

ELB Balancer

Amazon S3

Amazon CloudFront

Shift Some Load Around

• content to Amazon S3 and Amazon CloudFront

Web Instances

Move…Amazon Route 53

User

www.techarchday.fi

RDS DB InstanceActive (Multi-AZ)

Availability Zone

ELB Balancer

Amazon S3

Amazon CloudFront

Shift Some Load Around

• content to Amazon S3 and Amazon CloudFront

• session/state to Amazon DynamoDB

• DB caching to Amazon ElastiCache

Web Instances

Move…Amazon Route 53

User

DynamoDBElastiCache

www.techarchday.fi

Now that our web tier is much more lightweight, we can revisit the beginning

of our talk…

www.techarchday.fi

Auto Scaling!

www.techarchday.fi

Automatic resizing of compute clusters

Define min/max pool sizes

CloudWatch metrics drive scaling

On-demand or Spot instances

aws autoscaling create-auto-scaling-group--auto-scaling-group-name MyGroup--launch-configuration-name MyConfig--min-size 4--max-size 200--availability-zones us-west-2c, us-west-2b

Auto Scaling

www.techarchday.fi

Sunday Monday Tuesday Wednesday Thursday Friday Saturday

Typical Weekly Traffic to Amazon.com

Provisioned capacity

www.techarchday.fi

November

November Traffic to Amazon.com

Provisioned capacity

www.techarchday.fi

November Traffic to Amazon.com76%

24%November

Provisioned capacity

www.techarchday.fi

November Traffic to Amazon.com

November

www.techarchday.fi

Auto Scaling lets you do this!

www.techarchday.fi

Users > 500,000+

Availability Zone

Amazon Route 53

User

Amazon S3

Amazon CloudFront

Availability Zone

ELB Balancer

DynamoDB

RDS DB Instance Read Replica

Web Instance

Web Instance

Web Instance

ElastiCache RDS DB Instance Read Replica

Web Instance

Web Instance

Web Instance

ElastiCacheRDS DB Instance Standby (Multi-AZ)

RDS DB InstanceActive (Multi-AZ)

www.techarchday.fi

There are further improvements to be made

in breaking apart our web/app layer

www.techarchday.fi

The Service Oriented Architecture

www.techarchday.fi

The Service Oriented Architecture

www.techarchday.fi

Service Oriented Architecture

• Move services into their own tiers.– Treat them separately and

scale them independently.

• Amazon and AWS do this extensively!

• It offers flexibility and greater understanding of each component

www.techarchday.fi

Don’t Re-invent the Wheel

Leverage the AWS pre-built services.

• Email• Queuing• Transcoding• Search• Databases• Monitoring• Metrics• Logging• Compute• …

Amazon CloudSearch Amazon SQSAmazon SNS

Amazon Elastic TranscoderAmazon SWFAmazon SES

AWS Lambda

www.techarchday.fi

Users > 1 million+

Reaching a million and above is going to require some bit of all the previous things:

– Multi-AZ

– Elastic Load Balancing between tiers

– Serving content smartly (Amazon S3/CloudFront )

– Caching of DB

– Moving state off tiers that auto scale

– Auto Scaling

– Service Oriented Architecture

www.techarchday.fi

Users > 1 million+

RDS DB InstanceActive (Multi-AZ)

Availability Zone

ELB Balancer

RDS DB Instance Read Replica

RDS DB Instance Read Replica

Web Instance

Web Instance

Web Instance

Web Instance

Amazon Route 53

User

Amazon S3

Amazon CloudFront

DynamoDB

Amazon SQS

ElastiCache

WorkerInstance

WorkerInstance

Amazon CloudWatch

Internal AppInstance

Internal AppInstance Amazon SES

Lambda

www.techarchday.fi

Users > 5 million - 10 million

• You’ll potentially start to run into issues with your database around contention on the write master.

• How can you solve it?– Federation—splitting into multiple DBs based on function

– Sharding—splitting one dataset up across multiple hosts

– Moving some functionality to other types of DBs (NoSQL, Graph)

www.techarchday.fi

In Summary…

• Multi-AZ your infrastructure.

• Make use of self-scaling services—ELB, Amazon S3, Amazon SNS, Amazon SQS, Amazon SWF, Amazon SES, and more.

• Build in redundancy at every level.

• Use Auto Scaling once you’re ready for it.

• Split tiers into individual services (SOA).

• Select the right database architecture.

www.techarchday.fi

Putting all this together means we should now

easily be able to handle 10+ million users!

www.techarchday.fi

To Infinity...

www.techarchday.fi

Thank You!