AWS re:Invent 2016: Migrating Enterprise Messaging to the Cloud (ENT217)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Trevor Dyck, Senior Product Manager, AWS Messaging

Quinn Easterbrook, Director Platform Engineering, Capital One

November 30, 2016

Migrating Enterprise Messaging

to the Cloud

Agenda

Enterprise messaging overview

• Enterprise messaging challenges

• Cloud hosted messaging

• AWS messaging—Amazon SQS and Amazon SNS

• New: FIFO queues

Capital One use case

• Migrating retail application messaging to the cloud

What is “enterprise messaging ”?

Corporate data center

Inventory

CRM

Ordering

Front end

Ordering

Back endMoM

“Message-oriented middleware” (MoM)

Examples: IBM MQ, TIBCO

Active MQ, RabbitMQ (open source)

Messaging patterns

• Message queuing

• Publish-subscribe (pub-sub)

Message queueing

• Asynchronous

• Point-to-point

Producer

Consumer

Consumer

Queue

Publish-subscribe (pub-sub)

• Broadcast

• Point-to-multipoint

Publisher Subscriber

Subscriber

Topic

Subscriber

Use cases for message queues

Decouple application components or

microservices

Application integration

Batch and burst processing

Decouple application components or

microservices

• Web tier instances create work, worker instances

complete it

• Scale and manage tiers separately

• More resilient to failure vs direct RESTful communication

Application integration

• Applications exchange information asynchronously

• Apps can be independent, fault-tolerant

• Allows apps to be in different environments (OS, language)

Batch and burst processing

• Resilient to spikes in traffic

• Perform work only as fast as necessary to lower costs

• Never lose data

Challenges: message-oriented middleware (MoM)

Operational overhead

Capacity planning

Reliability

Cost

AWS managed messaging

services

Amazon Simple Queue Service (Amazon SQS)

• Fast, reliable, scalable, fully managed queue service

• AWS SDK or Java Message Service (JMS) APIs

Amazon Simple Notification Service (Amazon SNS)

• Fast, reliable, scalable fully managed pub-sub service

• Use topics to fan out messages to:

• Amazon SQS queues

• HTTP endpoints (web servers)

• AWS Lambda functions

• Mobile push, SMS, and email

Amazon SQS: key features

• Persistent message queue with high durability

and availability

• Messages are stored across multiple AZs

• Messages retained until deleted— up to 14 days

• Nearly unlimited throughput

Amazon SQS: key features

• Amazon CloudWatch metrics and alerts

• Message payloads up to 256 KB (5 TB using

AmazonS3)

• Message batching to increase throughput and

reduce cost

• Secure: uses AWS Identity & Access

Management (IAM) and HTTPS/TLS

No infrastructure to manage

Focus on business logic, not infrastructure.

Just write your code. Amazon SQS handles:

Capacity

Scaling

Deployment

Fault tolerance

Monitoring

Logging

Security

New! FIFO (first-in, first-out) queues

• Message ordering is strictly preserved

• Exactly once processing

• Message groups allow multiple ordered streams within a

single queue

• Essentially the same features and API as standard SQS

queues

• FIFO queues are limited to 300 transactions per

second (TPS)

• Standard SQS queues provide unlimited throughput

FIFO queues - ordering

ABC

Producer Consumer

FIFO queues – message groups

A

Producer

ConsumerGroup 1

ConsumerSendMessage (A, MessageGroupID=1)

FIFO queues – message groups

A

Producer

Consumer

X

Group 1

Group 2

ConsumerSendMessage (X, MessageGroupID=2)

FIFO queues – message groups

A

Producer

Consumer

XY

Group 1

Group 2

ConsumerSendMessage (Y, MessageGroupID=2)

SendMessage (B, MessageGroupID=1)

B

FIFO queues – message groups

A

Producer

Consumer

X

Group 1

Group 2

Consumer

ReceiveMessage

Y

A

X

Invisible

B

FIFO queues – message groups

Producer

Consumer

X

Group 1

Group 2

Consumer

ReceiveMessage

Y

X

B

FIFO queues – message groups

Producer

ConsumerGroup 1

Group 2

Consumer

ReceiveMessage

Y

Y

B

FIFO queues – message groups

Producer

ConsumerGroup 1

Group 2

Consumer

ReceiveMessage

Y

Y

B

B

FIFO Queues - deduplication

FIFO queues prevent duplicate messages

On send:

• For example, if producer retries before SQS acknowledges,

want to avoid a duplicate

• FIFO queues include MessageDeduplicationID for

duplicate detection

• Alternatively, content-based deduplication is also available

FIFO queues – deduplication

Producer

Consumer

ConsumerSendMessage (A, MsgDedupID = foo)

A

FIFO Queue

FIFO queues – deduplication

Producer

Consumer

Consumer

A

ReceiveMessage

FIFO Queue

Dedup buffer

A

FIFO queues – deduplication

Producer

Consumer

ConsumerSendMessage (A, MsgDedupID = foo)

A

Retry:

FIFO Queue

Dedup buffer

A

FIFO queues – deduplication

Producer

Consumer

Consumer

FIFO Queue

Dedup buffer

A

OK

FIFO queues – exactly-once processing

On receive:

• If consumer reads but does not process and delete a

message (for example, network failure)

• SQS will make the message available to another consumer

• If the consumer detects a failed ReceiveMessage action, it

can retry as many times as necessary, using the same ReceiveRequestAttemptId

FIFO queues – exactly-once processing

A

Producer

Consumer

Consumer

ReceiveMessage

A

(AttemptID = bar)

BC

FIFO Queue

FIFO queues – exactly-once processing

Producer

Consumer

Consumer

ABC

FIFO Queue

FIFO queues – exactly-once processing

A

Producer

Consumer

Consumer

ReceiveMessage

A

(AttemptID = bar)

BC

FIFO Queue

FIFO queues – pricing

• Low cost, per–API request pricing. Pay only for what you

use, no upfront costs.

• FIFO queues are priced at $0.50 per million requests.

• In conjunction, the price of SQS standard queues was

reduced to $0.40 per million requests.

• First 1 million SQS requests per month are free.

Move from risk-laden

upfront expense to

flexible variable expense

Stop guessing at

capacity planning

Go global in minutes

Hosted messaging: harness the cloud

Remove complicated

infrastructure management

Enterprises using AWS messaging services

BMWJohnson &

Johnson

Nordstrom

Migrating retail application messaging

to the cloud

A leading diversified bank with $339.1 billion in assets, $235.8 billion in loans and $221.1 billion in deposits1

• 8th largest bank based on U.S. deposits2

• 5th largest retail depository institution in metro New York3

• Largest consumer and commercial banking institution headquartered in the Washington, DC, region

• 4th largest credit card issuer in the U.S.4

• The 3rd largest issuer of small business credit cards in the U.S.5

• The 4th largest financial institution auto loan originator6

• Largest U.S. direct bank7

Major operations in 15 U.S. cities, Canada, U.K.

More than 65 million customer accounts, and 45,000 associates

A FORTUNE 500 Company - #112

Numerous recent awards including:

• Named to 100 Best Companies to Work For by FORTUNE Magazine

• Best Places to Work for LGBT Equality by Human Rights Campaign

• Received J.D. Power & Associates Call Center Certification

• Aon Hewitt’s Top Companies for Leaders

• Named to Working Mother’s 100 Best Companies list & Best Companies for Hourly Workers

• Ranked #10 on Military Times’ 2016 “Best for Vets”

• Recipient of the Secretary of Defense Employer Support Freedom Award

Capital One at a glance

1) Source: Company reported data as of Q2’16

2) Source: FDIC, Domestic deposits ranking as of Q2’16

3) Source: FDIC, June 2015, deposits capped at $1B per branch

4) Source: Company-reported domestic credit card outstandings, Q2’16,

5) Source: The Nilson Report, Issue 1089, June 2016

6) Note: Financial institutions includes banks & specialty finance lenders,

Source: AutoCount, FY 2015

7) Source: FDIC, company reports as of Q2’16

Retail messaging cloud migration

Benefits and cost saving associated with moving to the cloud

Focus on iterative improvements

• Maintaining existing architecture while providing modern components

• Provide integration points that work with existing applications

• Minimalize code changes where possible

Areas of upgrade

• Modernizing message queuing with Amazon SQS vs. self-managed systems

• Migrating hardware systems to Amazon EC2 with fully-automated build-out

• Leveraging Auto Scaling for resiliency and usage-based model

Current queue system

JBOSS JMS message queues used for legacy applications

JBOSS has been relatively stable, but has challenges:

• Downtime related to maintenance on servers running the queues

• Upgrades to the JMS client and server due to recoding efforts required

• Database system patching and issues with tablespace growth

• Visibility into queues and performance with homegrown monitoring

Challenges with queue administration and segregation

• Segmentation among applications required explicit separation of applications

and queues

Excessive overhead to administer multiple servers:

• 10 JBOSS JMS queue servers to 5 applications

Existing queue integration

2

1. User submits request to Web APP2. Web App sends message to Jboss MQ3. Middleware clusters poll Jboss MQ for new messages.4. Middleware sends Queries to Backend Databases and 3rd party App Vendors.5. Middleware sends the reply message to the application topic in JbossMQ6. Jboss MQ topic pushes the message back to the Web App.

Users

1

3

4

6

3rd Party Apps

5

CustDB1

CustDB2

Web App Jboss MQ Middleware Tomcat

Current queue system is complex

• Queues are distinct to each server,

applications lack visibility

• Instances are provisioned based on

theoretical load not actual load

• Requires timely build of supporting

infrastructure

• Multiple queues on each server make it

difficult to find failures

• Inability to identify bottlenecks and

resolve issues across systems

• Significant effort to manage multiple costly

systems and administer services

SQS selection criteria

Simplified administration

• Dynamic queue creation on startup of applications

• Requirement for patching and database maintenance removed

• Increased monitoring with Amazon CloudWatch built-in

Utilization patterns govern the number of queues required

• Static server builds not required for application demand

Required to be highly secure

• SQS provides TLS1.2 connectivity and strong authentication

Flexibility in queue types

• Can utilize FIFO as well as standard queues to allow for more performance

• FIFO use cases for ordering on pickup and classic for variable timed responses

SQS selection criteria

Ease of integration

• With small client changes can easily integrate into existing application

• Utilize existing JMS coding for application queues

Reliability/scaling

• Ability to scale as the applications grow without having to pre-plan sizing

• Inherent reliability with Multi-AZ deployments

Capacity planning

• Scaling of queues/resources based on when they are used vs. always on

Infrastructure simplification

• Elimination of costly physical systems with reliable serverless services

Private beta and SQS team interactions

Direct engagement with product team to develop features and solutions that

meet our specific needs

Successful process to collaborate and achieve key goals

• Examples: FIFO and exactly once processing

Exchange feedback on how solutions will work and solve complex issues

• Example: Solve for queue queries for message interaction reliability

Application queues

Existing New

New SQS Integration

RetailBank

CustDB1

AppSendQueue

2

1. User submits request to Web App.2. Web App instance sends message to Shared AppSendQueue for Middleware.3. Middleware Instances polls the AppSendQueue for new messages.4. Intra-Cluster message traffic.5. Middleware sends Queries to Backend Databases and 3rd party Vendors.6. Middelware sends reply message to the Instance Specific Reply Queue.7. WebApp Instance polls it’s specific reply queue for messages and correlates messages with initial request.

On-Prem

Users

Auto Scaling

Middleware

MW01Queue

MW02Queue

MW03Queue

Inst01RplyQueue

CustDB2

1

3

4

5

67

3rd Party Apps

5

Inst02RplyQueue

Inst0nRplyQueue

Auto Scaling

WebTop AppASG

WebTopInstance01

WebTopInstance02

WebTopInstance0n

7

7

6

6

FIFO

SQS migration wrap-up

Delivers significant benefit to overall migration efforts

• Lets us leverage existing systems without complex development

• Enables a seamless migration with limited overhead and administration

• Exceeds overall goals of resiliency and visibility

• Delivers future-proof solution that will grow with our needs

Summary

• Messaging is a common requirement in enterprise architectures

• Managed messaging services remove administrative overhead & “pain”

• FIFO queues enable more use cases

• AWS is investing in more key enterprise messaging features…

stay tuned!

Thank you!

Remember to complete

your evaluations!