Amazon Simple Queue Service(SQS) Amazon Simple Notification Service(SNS) AWS Black Belt Tech Webinar 2015 アマゾンデータサービスジャパン株式会社 ソリューションアーキテクト 篠原 英治 201534
Amazon Simple Queue Service(SQS)Amazon Simple Notication Service(SNS)
AWS Black Belt Tech Webinar 2015
201534
! NameEiji Shinohara ( )
! RoleSolutions Architect for StartupsAmazon CloudSearch Subject Matter Expert
Who am I?
Amazon Simple Queue Service (SQS) Amazon Simple Notication Service (SNS) SQSSNS
Amazon Simple Queue Service (SQS) Amazon Simple Notication Service (SNS) SQSSNS
Amazon Simple Queue Service (SQS)
https://aws.amazon.com/blogs/aws/amazon_simple_q/
2006713
Message Queue
MQ http://ja.wikipedia.org/wiki/
Amazon SQSPullMQ SQS
polling
Amazon SQSAWS : / : / : : +
Massive Message Processing with Amazon SQS and Amazon DynamoDB (ARC301) | AWS re:Invent 2013
SQS: Loosely Coupled
: :
150
Amazon SQS
SQS: Loosely Coupled Amazon SQSHappy
OK
JavaSQSAWS SDK
2
JavaPHP
PHP
Amazon SQS
Management Console
Amazon SQS
Management Console
Amazon SQS
Management Console :
Amazon SQS
Management Console
Amazon SQS
Management Console
Amazon SQS
Management Console :
Amazon SQS
Management Console - : Message Body
Amazon SQS
Management Console -
Amazon SQS
Management Console -
Amazon SQS Queue URL
URL https://.queue.amazonaws.com/ID/
Message ID ID
Receipt Handle Visibility
Visibility
Amazon SQS
AWS CLI
$ aws sqs create-queue --queue-name blackbelt2015{ QueueUrl: https://.queue.amazonaws.com/ID/blackbelt2015"}
Amazon SQS
AWS CLI :
$ aws sqs list-queues{ "QueueUrls": [ https://.queue.amazonaws.com/ID/blackbelt2015" ]}
Amazon SQS
AWS CLI
$ aws sqs send-message --queue-url https://.queue.amazonaws.com/ID/blackbelt2015 --message-body 'Hello World!'{ "MD5OfMessageBody": "ed076287532e86365e841e92bfc50d8c", "MessageId": "2db5a1de-8bc5-4976-9b0c-68749bee1a"}
Amazon SQS
AWS CLI
$ aws sqs receive-message --queue-url https://.queue.amazonaws.com/ID/blackbelt2015{ "Messages": [ { "Body": "Hello World!", ReceiptHandle: AQEBv3p27X8o44Nt4J0 +8lujWoy6Ks=", "MD5OfBody": "ed076287532e86365e841e92bfc50d8c", "MessageId": "2db5a1de-8bc5-4976-9b0c-68749bee1a" } ]}
Amazon SQS
AWS CLI
aws sqs delete-message --queue-url https://.queue.amazonaws.com/ID/blackbelt2015 --receipt-handle "AQEBv3p27X8o44Nt4J0 +8lujWoy6Ks="
Amazon SQS AWS SDK
AmazonSQS sqs = new AmazonSQSClient();CreateQueueRequest createQueueRequest = new CreateQueueRequest("MyQueue");String myQueueUrl = sqs.createQueue(createQueueRequest).getQueueUrl();
sqs.sendMessage(new SendMessageRequest(myQueueUrl, Hello World! "));
Amazon SQS AWS SDK
ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest(myQueueUrl);List messages = sqs.receiveMessage(receiveMessageRequest).getMessages();for (Message message : messages) {
// }
String messageRecieptHandle = messages.get(0).getReceiptHandle();sqs.deleteMessage(new DeleteMessageRequest(myQueueUrl, messageRecieptHandle));
Amazon SQS AWS SDK
AWS SDK20153
Java http://aws.amazon.com/jp/sdk-for-java/ Ruby http://aws.amazon.com/jp/sdk-for-ruby/ Python http://aws.amazon.com/jp/sdk-for-python/ .NET http://aws.amazon.com/jp/sdk-for-net/ PHP http://aws.amazon.com/jp/sdk-for-php/ JavaScript http://aws.amazon.com/jp/sdk-for-browser/ Node.js http://aws.amazon.com/jp/sdk-for-node-js/ Go(Coming Soon!) https://github.com/awslabs/aws-sdk-go
sqs.deleteQueue(new DeleteQueueRequest(myQueueUrl));
Amazon SQS
(SQS) 100 SQS1000.476 USD
1GB/0 USD 350TB /0.120 USD GB 350TB SQSEC2
Amazon SQS Visibility Timeout + EC2 Spot Instance API10/ Long PollReceive Message
http://www.slideshare.net/AmazonWebServices/advanced-topics-session-4-architecting-forhighavailability
Amazon SQS
Amazon SQS Visibility Timeout
1
2
3
Amazon SQS
Amazon SQS Visibility Timeout
1
1
2
3
Amazon SQS
Amazon SQS Visibility Timeout
30(12)23(in Flight)
1
2
3
Amazon SQS
Amazon SQS Visibility Timeout
1
1
2
3
Amazon SQS Amazon EC2 Spot Instances
Amazon EC2
201538
>= < 1
Amazon SQS Visibility Timeout + EC2 Spot Instances
Spot Instance1Visibility Timeout
:
Amazon SQS
1
2
3
Amazon SQS /1MAX10
SQS
Amazon SQS
1
2
3
1
2
3
x 10
x 10
x 10
x 10
x 10
x 10
Amazon SQS /1MAX10
: AWS CLI send-message-batch $ aws sqs send-message-batch--queue-url https://.queue.amazonaws.com/ID/blackbelt2015 --entries '[{ "Id": "01", "MessageBody": "test message 01" },{ "Id": "02", "MessageBody": "test message 02" },]{ "Successful": [ { "MD5OfMessageBody": "fa27aa462b939f0c8cf67189198f1b2d", "Id": "01", "MessageId": "041d1604-1c80-4440-90bf-598db00cf752" }, ...
Amazon SQS /1MAX10
: AWS CLI receive-message --max-number-of-messages(110)
$ aws sqs receive-message--queue-url https://.queue.amazonaws.com/ID/blackbelt2015 --max-number-of-messages 10{ "Messages": [ { "Body": "test message 01", "ReceiptHandle": "AQEB 7g/jA=", "MD5OfBody": "fa27aa462b939f0c8cf67189198f1b2d", "MessageId": "041d1604-1c80-4440-90bf-598db00cf752" },
Amazon SQS
AWS SDK for Java AmazonSQSAsyncClient 10
maxBatchOpenMs: maxBatchSize: 1
Amazon SQS Long Polling
SQSReceive Message
(020)
Amazon SQS Long Polling Short Polling
Long Polling (CPU, ) Polling
Receive MessageShort Polling Long Polling
Polling
Polling
Polling Polling
Webinar SQS Message Attributes AWS CloudTrail Delete All Messages in SQS Queue SQS Client Library for Java Messaging Service (JMS)
Webinar Message Attributes 201456
Body10 NameType(String/Number/Binary)Value
Type(: oat, int)
Webinar AWS CloudTrail 2014716
APIS3 CloudTrail20147
Webinar
Delete All Messages in SQS Queue 2014128 PurgeQueue
Webinar
SQS Client Library for Java Messaging Service (JMS) 20141229 JMS: JavaAPI SQSJMS
JMS
Amazon SQS Amazon SQS
(1) SQS/
Amazon SQS
46014
In Flight(Visibility Timeout) 120,000In Flight OverLimit
256KB 2013664KB256KB
S3
Delay Queue Message Timers
Dead Letter Queue SQS
Elastic Beanstalk: Worker Tier
AWS Identity and Access Management (IAM)
CloudWatch
Delay Queue
0900
Message Timers
0900 Message TimersDelay Queue
Send Message Request
Delay Queue 30
Message Timers 60
Dead Letter Queue
(11000)(Dead Letter Queue)
0900
Dead Letter Queue
AWS Elastic Beanstalk: Worker Tier
SQS Auto Scaling
Sqsd(deamon)
Elastic Beanstalk Application
http://localhost:80/xxx
EC2 Instance
Auto Scaling group
CloudWatch
Auto Scaling
Job Observer
http://aws.clouddesignpattern.org/index.php/CDP:Job_Observer AWS
http://aws.typepad.com/aws_japan/2015/01/auto-scaling-with-sqs.html
AWS Identity and Access Management (IAM)
/ { "Version":"2012-11-05", "Id":"cd3ad3d9-2776-4ef1-a904-4c229d1642ee", "Statement" : [ { "Sid":"1", "Eect":"Allow", "Principal" : { "aws": "111122223333" }, "Action":["sqs:SendMessage","sqs:ReceiveMessage"], "Resource": "arn:aws:sqs:us-east-1:444455556666:queue2", "Condition" : { "IpAddress" : { "aws:SourceIp":"10.52.176.0/24" }, "DateLessThan" : { "aws:CurrentTime":"2009-06-30T12:00Z
CloudWatch
NumberOfMessageSent
SentMessageSize
NumberOfMessageReceived ReceiveMessage
NumberOfEmptyReceives ReceiveMessage
NumberOfMessagesDeleted
ApproximateNumberOfMessageDelayed DelayDelay QueueDelay
ApproximateNumberOfMessageVisible
ApproximateNumberOfMessageNotVisible visibility winodowend
Amazon Simple Queue Service (SQS) Amazon Simple Notication Service (SNS) SQSSNS
Amazon Simple Notication Service (SNS)
Amazon SNS
Publish
1.Topic
2.
Amazon SNS
Publisher Topic
HTTP(S)
SQS
Mobile Push
Subscriber
Amazon Simple Notication Service (SNS) Mobile Push(2015311)AWS Black Belt
Tech Webinar
Amazon SNS Mobile Push
Apple Devices
Google Devices
Amazon Kindle Fire Devices
Amazon SNS Amazon SNS :
: 100 Email/Email-JSON: 1,000 HTTP/HTTPS: 100,000 Simple Queue ServiceSQS:
(64 KB 1) : 1000.5 USD Email/Email-JSON: 100,0002 USD HTTP/HTTPS: 1000.6 USD Simple Queue ServiceSQS:
SMS
Amazon SNS Management Console Mobile Services
Amazon SNS1. Topic
2. TopicSubscribe3. TopicPublish
Amazon SNS1. Topic
TopicARN(Amazon Resource Name)
Amazon SNS2. TopicSubscribe
EndpointConrm
Amazon SNS3. TopicPublish
Publish
(Email)
Amazon Simple Notication Service (SNS)
AWS
Amazon CloudWatch Billing Alert
Amazon SES Bounce/Complaint
Amazon S3
Amazon Elastic Transcoder /
Amazon SNS Amazon CloudWatchBilling Alert
Amazon SNS
Amazon SESBounce/Complaint
Amazon SNS
Amazon S3
Amazon SNS Amazon Elastic Transcoder/
HTTPS enhancements Server Name Indication (SNI) Basic Digest
Message Attributes Time To Live (TTL)
Webinar
HTTPS enhancements 2014428 Server Name Indication (SNI)
SNISSL/TLSSSLIP
http://ja.wikipedia.org/wiki/Server_Name_Indication Basic Digest
HTTP POST HTTPS URL http://www.rfc-editor.org/info/rfc2617
Webinar
Message Attributes 2014612 String, Number, Binary Mobile PushReserved Message Attributes
Baidu MPNS WNS
Webinar
TTL 2014710 Time To Live
Webinar
Amazon Simple Queue Service (SQS) Amazon Simple Notication Service (SNS) SQSSNS
Cloud Design Pattern: Fanout http://aws.clouddesignpattern.org/index.php/CDP:Fanout
SQSSNS
S3 RDS DynamoDB
SQSSNS
WebSNS
SNSSQS
Amazon S3
Amazon RDS
DynamoDB
SQS
S3 RDS DynamoDB
SQSSNS
WebSNS
SNSSQS
Amazon S3
Amazon RDS
DynamoDB
SQS
SNSSQS (64 KB 1)
Simple Queue ServiceSQS:
SQSSNS
WebSNS
SNSSQS
Amazon S3
Amazon RDS
DynamoDB
SQS
Amazon Simple Queue Service (SQS) Amazon Simple Notication Service (SNS) SQSSNS
Amazon SQS
Amazon SNS
AWS
Amazon SQS Document
http://aws.amazon.com/jp/documentation/sqs/ Amazon SQS FAQ
http://aws.amazon.com/jp/sqs/faqs/ Amazon SNS Document
http://aws.amazon.com/jp/documentation/sns/ Amazon SNS FAQ
http://aws.amazon.com/jp/sns/faqs/ AWS
http://aws.clouddesignpattern.org/index.php Massive Message Processing with Amazon SQS and Amazon DynamoDB
(ARC301) | AWS re:Invent 2013 https://www.youtube.com/watch?v=RJCpjRTvo_w
Webinar
AWS http://aws.amazon.com/jp/aws-jp-introduction/
Twitter/FacebookAWS
@awscloud_jp
http://on.fb.me/1vR8yWm