Page 1
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
AWS CloudFormation Under the Hood
Adam Thomas, Amazon Web Services
DJ Edwards, Amazon Web Services
November 14, 2013
Page 2
So, what is CloudFormation?
Page 3
This talk will not answer that question
• DMG201 - Zero to Sixty: AWS
CloudFormation – Has already happened, but will be available online
• Hands-on Labs – Working with CloudFormation
– Launching and Managing a Web Application with
CloudFormation
– Creating an Amazon Virtual Private Cloud (VPC) with
CloudFormation
Page 4
This talk will answer these questions:
• What is a custom resource?
• What can they do for me?
• How do I write one for myself?
• What’s new in cfn-init?
Page 6
What can custom resources do?
• Add New Resources
• Interact with the CloudFormation Workflow
• Inject dynamic data into a stack
• Extend the capabilities of existing resources
Page 7
What is a custom resource?
• An SNS topic…
• …hooked up to a service that can: – Respond to JSON messages from CloudFormation
– Manage the lifecycle of resources
Page 8
How are custom resources defined? "myCustomResource" : {
"Type": "Custom::MyCustomResource",
"Version" : "1.0",
"Properties" : {
"ServiceToken": "arn:aws:sns:us-east-1:84969EXAMPLE:CRTest",
"CustomProperty" : "foo"
}
}
Page 9
How are custom resources defined? "myCustomResource" : {
"Type": “Custom::MyCustomResource",
"Version" : "1.0",
"Properties" : {
"ServiceToken": "arn:aws:sns:us-east-1:84969EXAMPLE:CRTest",
"CustomProperty" : "foo"
}
}
Page 10
How are custom resources defined? "myCustomResource" : {
"Type": “Custom::MyCustomResource",
"Version" : "1.0",
"Properties" : {
"ServiceToken": "arn:aws:sns:us-east-1:84969EXAMPLE:CRTest",
"CustomProperty" : "foo"
}
}
Page 11
What can custom resources do?
• Add New Resources
• Interact with the CloudFormation Workflow
• Inject dynamic data into a stack
• Extend the capabilities of existing resources
Page 12
Adding New Resources
• Something that can be Created, Updated,
and/or Deleted
• Can be a software resource – Database schema, Docker container
Page 13
Meet Steve
• Steve loves RDBMS
• The schema is very
important to Steve – it
defines his application
• Running SQL scripts by
hand is Steve’s worst
nightmare
Page 14
Steve’s requirements
• The Template should define the schema explicitly
• The schema should be updated by updating the stack
• If the update fails, the schema should roll back
Page 15
Steve’s solution
• Steve is very familiar with
Liquibase
• Liquibase supports JSON
formatting!
• Steve writes a custom
resource with inline JSON
schema
Page 16
DB Schema Template Snippet "appSchema" : {
"Type" : "Custom::DatabaseSchema",
"Properties" : {
"databaseChangeLog" : [{ "changeSet" : {
"id" : "1",
"author" : “adam",
"changes" :
[{ "createTable" : {
"tableName" : "person",
"columns" : …
Page 17
DB Schema Template Snippet "appSchema" : {
"Type" : "Custom::DatabaseSchema",
"Properties" : {
"databaseChangeLog" : [{ "changeSet" : {
"id" : "1",
"author" : “adam",
"changes" :
[{ "createTable" : {
"tableName" : "person",
"columns" : …
Page 19
What can custom resources do?
• Add New Resources
• Interact with the CloudFormation Workflow
• Inject dynamic data into a stack
• Extend the capabilities of existing resources
Page 20
Interacting with the CloudFormation
Workflow • Use custom resources as a hook into
create/update/delete workflows
• Built-in example: WaitCondition
• Can react to workflow, halt it, or fail it under
certain conditions
Page 21
Meet Frank
• Frank analyzes data
stored on EBS
• Frank uses
CloudFormation’s
Snapshot on Delete
feature to save his
analysis results
Page 22
Frank’s requirements
• Frank wants a consistent EBS snapshot when the stack is deleted
• Before CloudFormation attempts to detach his EBS volume, it should: – Cleanly shut down his
analysis service
– Unmount the volume
Page 23
Why is this a challenge?
• CloudFormation can detach volumes without
any issues – if you never mount them
• What CloudFormation does not do, it cannot
undo
• Custom resources let you model your steps
within the workflow
Page 24
Frank’s solution
• 3 simple bash scripts
• A “local” Custom
Resource – runs directly
on the instance
• Create and Update mount
the drive; Delete
unmounts it.
Page 25
Volume Mount Template Snippet “VolumeAttach" : {
"Type" : "AWS::EC2::VolumeAttachment",
"Properties" : …
},
"VolumeMount" : {
"Type" : "Custom::VolumeMount",
"Properties" : {
"Device" : “/dev/xvdh”,
“MountPoint” : “/mnt/analysis”
}
}
Page 26
Volume Mount Template Snippet “VolumeAttach" : {
"Type" : "AWS::EC2::VolumeAttachment",
"Properties" : …
},
"VolumeMount" : {
"Type" : "Custom::VolumeMount",
"Properties" : {
"Device" : “/dev/xvdh”,
“MountPoint” : “/mnt/analysis”
}
}
Page 27
Volume Mount Template Snippet “VolumeAttach" : {
"Type" : "AWS::EC2::VolumeAttachment",
"Properties" : …
},
"VolumeMount" : {
"Type" : "Custom::VolumeMount",
"Properties" : {
"Device" : “/dev/xvdh”,
“MountPoint” : “/mnt/analysis”
}
}
Page 28
Volume Mount Template Snippet “VolumeAttach" : {
"Type" : "AWS::EC2::VolumeAttachment",
"Properties" : …
},
"VolumeMount" : {
"Type" : "Custom::VolumeMount",
"Properties" : {
"Device" : “/dev/xvdh”,
“MountPoint” : “/mnt/analysis”
}
}
Page 29
Volume Mount Demo
Page 30
What can custom resources do?
• Add New Resources
• Interact with the CloudFormation Workflow
• Inject dynamic data into a stack
• Extend the capabilities of existing resources
Page 31
Injecting Dynamic Data into a Stack
• Parameters are standard route into a stack – Allow free-form user input
– Constrainable, but on a per-stack level
• Mappings are traditionally used to map human-
readable input to static values – AMI IDs, instance type architectures, regional URLs
Page 32
Injecting Data into a Stack
• Custom resources allow for centralized selection
logic
• Lookups in: – S3
– DynamoDB/RDS
– APIs (EC2.DescribeImages, etc)
– Third Party datastore
Page 33
Meet Bill
• Bill is the head of operations
at a large tech firm
• Each of Bill’s 44 services
must run on a fully validated
and tested AMI
• Bill keeps track of these
AMIs in a sweet multi-
tabbed Excel spreadsheet
Page 34
Bill’s requirements
• New AMIs should be rolled out centrally
• Bill does not want to edit the Mappings section of 44 templates for every release
• Bill wants to audit where AMIs are being used
Page 35
Bill’s solution
• A manifest of named,
approved AMIs stored in a
versioned S3 file
• A simple python script that
looks up the AMI ID by
region and os,
architecture, and version
Page 36
AMI Lookup Template Snippet "AMILookup": {
"Type": "Custom::AmiLookup",
"Properties": {
"os": "ubuntu",
"version": “13.04",
"arch": "64"
}
},
"WebServer": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId" : { “Ref" : “AMILookup” }
}
}
Page 37
AMI Lookup Template Snippet "AMILookup": {
"Type": "Custom::AmiLookup",
"Properties": {
"os": "ubuntu",
"version": "13.04",
"arch": "64"
}
},
"WebServer": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId" : { “Ref" : “AMILookup” }
}
}
Page 38
AMI Lookup Template Snippet "AMILookup": {
"Type": "Custom::AmiLookup",
"Properties": {
"os": "ubuntu",
"version": "13.04",
"arch": "64"
}
},
"WebServer": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId" : { “Ref" : “AMILookup” }
}
}
Page 40
What can custom resources do?
• Add New Resources
• Interact with the CloudFormation Workflow
• Inject dynamic data into a stack
• Extend the capabilities of existing resources
Page 41
Extending Resource Capabilities
• CloudFormation is concerned only with Create,
Update, and Delete
• Some services, like AutoScaling, have lifecycles
outside of these phases
• No place in template to encapsulate long-
running, resource-based business logic
Page 42
Meet Tom
• Tom manages a fleet of
virtual desktops in AWS
• Tom uses AutoScaling for
consistent fleet size
• Tom’s users use VNC to
connect to their virtual
desktops
Page 43
Tom’s requirements
• Servers should be named
using his clever, easy-to-
remember Simpsons
scheme
• Names should be
recycled as machines are
replaced
Page 44
Tom’s solution
• Python scripts respond to
Auto Scaling notifications
to manage Route53
records
• Names are managed in a
simple DynamoDB table
Page 45
Auto Scaled DNS Snippet (1 of 2) "DNSProcessor" : {
"Type": "Custom::DNSProcessor",
"Properties": {
"HostedZoneId" : { "Ref" : "HostedZone" },
"DNSPattern" : {"Fn::Join" : [".",[“{{simpsons_name}}", { "Ref" : "AWS::Region" }, “{{hosted_zone_name}}"]] }
}
},
Page 46
Auto Scaled DNS Snippet (1 of 2) "DNSProcessor" : {
"Type": "Custom::DNSProcessor",
"Properties": {
"HostedZoneId" : { "Ref" : "HostedZone" },
"DNSPattern" : {"Fn::Join" : [".",[“{{simpsons_name}}", { "Ref" : "AWS::Region" }, “{{hosted_zone_name}}"]] }
}
},
Page 47
Auto Scaled DNS Snippet (2 of 2) "WebServerGroup" : {
"Type" : "AWS::AutoScaling::AutoScalingGroup",
"Properties" : {
"NotificationConfiguration" : {
"TopicARN" : { "Fn::GetAtt" : ["DNSProcessor", “Topic"] },
"NotificationTypes" : [ "autoscaling:EC2_INSTANCE_LAUNCH","autoscaling:EC2_INSTANCE_TERMINATE"]
},
"Tags" : [{ "Key" : "ProcessorId",
"Value" : { "Ref" : "DNSProcessor" },
"PropagateAtLaunch" : false }]
}
}
Page 48
Auto Scaled DNS Snippet (2 of 2) "WebServerGroup" : {
"Type" : "AWS::AutoScaling::AutoScalingGroup",
"Properties" : {
"NotificationConfiguration" : {
"TopicARN" : { "Fn::GetAtt" : ["DNSProcessor", “Topic"] },
"NotificationTypes" : [ "autoscaling:EC2_INSTANCE_LAUNCH","autoscaling:EC2_INSTANCE_TERMINATE"]
},
"Tags" : [{ "Key" : "ProcessorId",
"Value" : { "Ref" : "DNSProcessor" },
"PropagateAtLaunch" : false }]
}
}
Page 49
Auto Scaled DNS Demo
Page 50
Building Your Own Custom Resource
• Write code to respond to Create, Update, and
Delete events
• Route Custom Resource SNS Topic to an SQS
Queue for maximum fault tolerance
Page 51
Can you give me a diagram?
Other resources access Custom Resource attributes via GetAtt and Ref
Stack workflow continues
CloudFormation processes JSON message and stores result
Custom Resource creates resource and returns JSON message
CloudFormation sends CREATE notification to Custom Resource
CloudFormation Stack Workflow starts building Custom Resource
Page 52
How about an architectural overview?
AWS CloudFormation
Custom Resource Topic SQS Queue
Region
Auto scaling Group
Custom Resource
Implementation
Page 53
Can you add VPC?
AWS CloudFormation
Custom Resource Topic SQS Queue
Region
Corporate Data center
Existing Service VPN Custom Resource
Implementation
Page 54
What makes for a good resource?
• Good resources are: idempotent – One unique request, n times == one unique response
• Immediately usable when complete
• Can be deleted cleanly from any state
• Represent one standalone piece of functionality – Embedded resources look convenient, but are hard to update
– Elastic Load Balancers embed Policies, which can depend on
each other, yet this is not modeled in the template
Page 55
You keep telling me it’s simple…
• It’s really simple if you use aws-cfn-resource-bridge
• Cross-platform hook-based daemon
• Simply supply scripts for Create, Update, and Delete
• Open source (Apache 2.0)
• Install or fork from https://github.com/aws/aws-cfn-resource-bridge
Page 56
Example Code
• All examples from this talk are available at
https://github.com/awslabs/aws-cfn-custom-
resource-examples
• Stealing from others is the easiest way to get
started – And the best way to use CloudFormation!
Page 58
cfn-init
• Simple library for “getting bits on the box”
• Install packages, download files, start services
• Works on Windows, Linux, and any platform with
Python 2.6, 2.7
Page 59
{{cfn-init}}
• Fn::Join can be hard to follow
• Many configuration files are largely boilerplate
• Files can process Mustache templates
• Simply add context
Page 60
Wordpress config "/var/www/html/wordpress/wp-config.php" : {
"content" : { "Fn::Join" : ["", [
"<?php\n",
"define('DB_NAME', '", {"Ref" : "DBName"}, "');\n",
"define('DB_USER', '", {"Ref" : "DBUser"}, "');\n",
"define('DB_PASSWORD', '", {"Ref" : "DBPassword" }, "');\n",
"define('DB_HOST', '", {"Fn::GetAtt" : [“MyDB", "Endpoint.Address"]},"');\n",
"define('DB_CHARSET', 'utf8');\n",
"define('DB_COLLATE', '');\n",
"define('AUTH_KEY', 'f@A17vs{ mO0}:&I,6SB.QzV`E?!`/tN5:~GZX%=@ZA%!_T0-]9>g]4ll6~,6G|R');\n",
Page 61
Wordpress config "/var/www/html/wordpress/wp-config.php" : {
"content" : { "Fn::Join" : ["", [
"<?php\n",
"define('DB_NAME', '", {"Ref" : "DBName"}, "');\n",
"define('DB_USER', '", {"Ref" : "DBUser"}, "');\n",
"define('DB_PASSWORD', '", {"Ref" : "DBPassword" }, "');\n",
"define('DB_HOST', '", {"Fn::GetAtt" : [“MyDB", "Endpoint.Address"]},"');\n",
"define('DB_CHARSET', 'utf8');\n",
"define('DB_COLLATE', '');\n",
"define('AUTH_KEY', 'f@A17vs{ mO0}:&I,6SB.QzV`E?!`/tN5:~GZX%=@ZA%!_T0-]9>g]4ll6~,6G|R');\n",
Page 62
Wordpress config "define('SECURE_AUTH_KEY', 'gTFTI|~rYHY)|mlu:Cv7RN]GQ^3ngyUbw;L0o!12]0c-ispR<-yt3qj]xjquz^&9');\n",
"define('LOGGED_IN_KEY', 'Jd:HG9M)1p5t2<v~+R-vd{p-Q*|*RB^&PUI{vIrydAEEiV!{HS{jN:nErCmLv`p}');\n",
"define('NONCE_KEY', '4aMj4KZV;,Gu7(B|qOCve[c5?*J5x1+x93i:Ey6hh/6jXh+V_{V4+hw!qE^d*U,-');\n",
"define('AUTH_SALT', '_Y_&8m)FH)Cns)8}Yb8b88KDSn:p1#p(qBa<~VW&Y1v}P.*9/8S8@P`{mkNxV lC');\n",
"define('SECURE_AUTH_SALT', '%nG3Ag41^Lew5c86,#zbN:yPFs.GA5a)z5*:Oce1>v6uF~D`,.o1pzS)F8[bM9i[');\n",
"define('LOGGED_IN_SALT', '~K<y+Ly+_Ww1~dtq>;rSQ^+{P5/k|=!]k%RXAF-Y@XMY6GSp+wJ5{(|rCzaWjZ%/');\n",
"define('NONCE_SALT', ',Bs_*Y9:b/1Z:apVLHtz35uim|okkA,b|Jt[-&Nla=T{<l_#D?~6Tj-.2.]FonI~');\n",
Page 63
Wordpress config "define('WPLANG' , '');\n",
"define('WP_DEBUG' , false);\n",
"$table_prefix = 'wp_';\n",
"if ( !defined('ABSPATH') )\n",
" define('ABSPATH', dirname(__FILE__) . '/');\n",
"require_once(ABSPATH . 'wp-settings.php');\n"
]] }
}
Page 64
{{cfn-init}} Wordpress Config “files” : {
"/var/www/html/wordpress/wp-config.php" : {
“source” : “https://github.com/FAKEPATH/wp-config.mustache”,
“context” : {
“DbEndpoint” : {“Fn::GetAtt” : [“MyDB”, “Endpoint.Address”]},
“DbName” : { “Ref” : “DbName” },
“DbUser” : { “Ref” : “DbUser” },
“DbPassword” : { “Ref” : “DbPassword” }
}
}
}
Page 65
{{cfn-init}} Wordpress Config “files” : {
"/var/www/html/wordpress/wp-config.php" : {
“source” : “https://github.com/FAKEPATH/wp-config.mustache”,
“context” : {
“DbEndpoint” : {“Fn::GetAtt” : [“MyDB”, “Endpoint.Address”]},
“DbName” : { “Ref” : “DbName” },
“DbUser” : { “Ref” : “DbUser” },
“DbPassword” : { “Ref” : “DbPassword” }
}
}
}
Page 66
{{cfn-init}} Wordpress Config “files” : {
"/var/www/html/wordpress/wp-config.php" : {
“source” : “https://github.com/FAKEPATH/wp-config.mustache”,
“context” : {
“DbEndpoint” : {“Fn::GetAtt” : [“MyDB”, “Endpoint.Address”]},
“DbName” : { “Ref” : “DbName” },
“DbUser” : { “Ref” : “DbUser” },
“DbPassword” : { “Ref” : “DbPassword” }
}
}
}
Page 67
Roleplaying
• cfn-init can use roles to download from S3
• Secured files are not just for proprietary code – Non-AWS credentials
– Private service endpoints
– Dynamic code (enabling or disabling features)
Page 68
Roleplaying Template Snippet “AWS::CloudFormation::Authentication” : {
“roleCreds” : {
“type” : “S3”,
“roleName” : “MyS3Role”
}
}
…
“files” : {
“/etc/secrets.txt” : {
“source” : “https://s3.amazonaws.com/mybucket/secrets.txt”,
“authentication” : “roleCreds”
}
}
Page 69
Roleplaying Template Snippet “AWS::CloudFormation::Authentication” : {
“roleCreds” : {
“type” : “S3”,
“roleName” : “MyS3Role”
}
}
…
“files” : {
“/etc/secrets.txt” : {
“source” : “https://s3.amazonaws.com/mybucket/secrets.txt”,
“authentication” : “roleCreds”
}
}
Page 70
Roleplaying Template Snippet “AWS::CloudFormation::Authentication” : {
“roleCreds” : {
“type” : “S3”,
“roleName” : “MyS3Role”
}
}
…
“files” : {
“/etc/secrets.txt” : {
“source” : “https://s3.amazonaws.com/mybucket/secrets.txt”,
“authentication” : “roleCreds”
}
}
Page 71
cfn-hup
• Not new, but not often used in samples
• Installed in same package as cfn-init
• Available as Linux and Windows service
• Listens for changes to the stack and runs scripts
when they occur – Usually just runs or re-runs cfn-init
Page 72
Custom Resources vs. cfn-hup
• Custom Resources require an SNS topic, and
usually an SQS queue
• cfn-hup cannot interact with CloudFormation
workflow – Workflow will not wait for cfn-hup
– cfn-hup cannot fail workflow
– cfn-hup cannot inject data into stack
Page 73
Summary
• Custom Resources let you extend
CloudFormation beyond the existing Resource
Library
• For more than just “things that can be created”
• cfn-init lets you use Mustache and Roles to
create simple, secure configuration
Page 74
Corner us in the Developer Lounge
Adam Thomas DJ Edwards
Page 75
Please give us your feedback on this
presentation
As a thank you, we will select prize
winners daily for completed surveys!
DMG303 - AWS CloudFormation Under the Hood