AWS Summit London 2014 - JUST EAT - High Availability and Rapid Change

© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

Customer Story: JUST EATDaniel Richardson, Director of Engineering

30 APRIL 2014

• 13 countries• 36,000 restaurants• Nearly 6 million active

accounts• Nearly 1000 orders a

minute at peak

What we use

Amazon EC2 Amazon EMR Amazon Route 53

Elastic LoadBalancing

AWS Direct Connect

Amazon VPC Auto Scaling

Amazon S3 Amazon EBS

Amazon Glacier

CloudFront

DynamoDB Amazon RDS Amazon SimpleDB

Amazon Redshift

CloudWatch AWS Data Pipeline AWS CloudFormatio

n

IAM Amazon SES

Amazon SNS Amazon SQS Amazon SWF

Engineering goals

High Availability Rapid Change

How we meet our goals

Teams

• Small• Independent• Responsible from

development to live• Focused on features for

consumers and restaurants

Architecture

• Small components• Loosely coupled• Highly available• Designed for failure• Infrastructure as code

Meeting our goals with AWS• We want small,

independent teams that own loosely coupled components

• Separate platform into many CloudFormation stacks

CloudFormation

• Create and manage a set of related AWS resources

• Define resources in template using JSON– Easy to inspect and validate– Easy to consume and generate– Easy to diff and source control

• CloudFormation creates a running stack from template and manages the lifecycle of resources

CloudFormation stacks and teams

Platform Services

Websites and Apps

APIs

Meeting our goals with AWS• We want teams to

take responsibility for the high availability of their components

• Make use of highly available building blocks

Auto Scaling

Amazon Route 53

Elastic LoadBalancing Amazon S3 Amazon

EBSAmazon Glacier

DynamoDB Amazon RDS

Amazon SES

Amazon SNS Amazon SQS

Inside a stack – Order API

Platform Services

Websites and Apps

APIs

Inside a stack – Order API

Auto Scaling group

eu-west-1a

eu-west-1b

eu-west-1c

Amazon S3 bucket

Amazon DynamoDB table

Amazon SQS queue

Amazon SNS topic

Elastic Load Balancer

Route53 hosted zone

Launching a stack – Order API

• CloudFormation template is generated as output• The template defines the entire stack

• CloudFormation generates resources defined in templateand manages their lifecycle

• Running stack

• Ruby command line utility takes information about thefeature as parameters

• Combines component-specific requirements with commonresources, standards and best practices

EnvMan

Template

Stack

Demo: High availability (…or how avoid having your Saturday night ruined)

Restaurant Devices

High availability

Auto Scaling group

eu-west-1a

eu-west-1b

eu-west-1c

Elastic Load BalancerPerforms health checks

JCT devices in restaurants

Active

Backup

Devices connect to Elastic IP Address

Let’s break something

High availability

Auto Scaling group

eu-west-1a

eu-west-1b

eu-west-1c

Elastic Load BalancerPerforms health checks

JCT devices in restaurants

Active

Backup

Devices connect to Elastic IP Address

High availability

Auto Scaling group

eu-west-1a

eu-west-1b

eu-west-1c

JCT devices in restaurants

Backup

StackMan

Elastic Load BalancerPerforms health checks

High availability

Auto Scaling group

eu-west-1a

eu-west-1b

eu-west-1c

JCT devices in restaurants

Active

BackupStackMan

Elastic Load BalancerPerforms health checks

Did it work?

Meeting our goals with AWS• We want teams that

can focus on building new features that customers will love

• Treat infrastructure as code

Order workflow

Order placed

Send to restaurant

Accepted

RefusedAnalysis

Good

Potential Issue

Email confirmatio

n

Send text message

Order workflow

Order placed

Send to restaurant

Accepted

RefusedAnalysis

Good

Potential Issue

Email confirmatio

n

Send text message

Amazon SNS Amazon SQS

Order workflow

OrdersAnalysis

Potentialissue

Good

Send torestaurant

Accepted

Refused

Confirmationemail

Textmessage

Order workflow

OrdersAnalysis

Potentialissue

Good

Send torestaurant

Accepted

Refused

Confirmationemail

Textmessage

Order workflow

OrdersAnalysis

Potentialissue

Good

Send torestaurant

Accepted

Refused

Confirmationemail

Textmessage

Publish message to topic

Other parts of the platform can subscribe

JustSaying

• Open source library for publishing and consuming events / messages over SNS

• Create SNS topics at runtime in code but expressed as business rules

• Download next month: http://github.com/justeat/

JustSayingpublic class OrderAccepted : Message { public OrderAccepted(int orderId) { OrderId = orderId; } public int OrderId { get; private set; }}

var publisher = FluentNotificationStack.Register(configuration => { configuration.Component = Component.OrderValidator; configuration.PublishFailureReAttempts = 3; }) .WithSnsMessagePublisher<OrderAccepted>("OrderProcessing");

publisher.Publish(new OrderAccepted(7349753));

AWS allows us to meet out goals

Engineering goals• Small, independent teams

that own loosely coupled components

• Teams take responsibility for high availability of their components

• Teams that can focus on building new features that customers will love

AWS

• Separate platform into many CloudFormation stacks

• Make use of highly available building blocks

• Treat infrastructure as code

Find out more

• Blog – http://tech.just-eat.com• Open source - https://github.com/justeat• Yes, we’re hiring - http://tech.just-eat.com/jobs• Come and talk to us later – look for the t-shirts