UFO Tung Nguyen Jan 2017
UFOTung Nguyen
Jan 2017
About Me
I’ve done a little bit of everything. Both dev and ops.
@tongueroo - Twitter.https://medium.com/@tongueroo - Blog.tongueroo.com - Personal Site.
lono - CloudFormation template generator.
jack - Manage ElasticBeanstalk environments.
thor_template - used this to generate this tool.
ufo - ship docker images to ecs. What this talk is about.
These tools are in Ruby.More: http://tongueroo.com/projects/
Other Tools
Why?Wanted get my hands dirty with ECS.Wanted more control over the ECS task definition.
Wanted to automated the building of the docker image as part of building task definition.
Ending up using it on my side projects.
ECS Introduction - TermsTask definition - blueprint for docker container.
Task - Actual running container.Service - Long running tasks.Cluster - Logic group of EC2 Container instances.
Container Instance - EC2 instance that is part of an ECS Cluster.
ECS Introduction - Terms
Steps for Deploying on ECS1.Build and push docker image2.Build and register task definition 3.Update the ECS service (create the service if needed)
Google Search: “deploying to ecs”silinternational/ecs-deploy - pulls down existing task definition.
CircleCI deploy.sh example - nice and simple easy to build from.
Empire - heroku like opensource SaaS from Remind engineering. A little more than I need but cool project.
Programmer Virtue #1: Laziness -> Research
Wanted more control over the task definition template.
Would be nice if the task definition could be reused for common and similar processes: web, clock, worker.
More Control
Task Definition Similarities$ cat ufo/output/hi-web.json { "family": "hi-web", "containerDefinitions": [ { "name": "web", "image": "tongueroo/hi:ufo-2016-12-04T17-54-23-2d1634d", "cpu": 128, "memoryReservation": 256, "portMappings": [ { "containerPort": "3000", "protocol": "tcp" } ], "command": [ "bin/web" ], "environment": [ { "name": "ADMIN_PASSWORD", "value": "secret" } ], "essential": true } ]}$
$ cat ufo/output/hi-worker.json { "family": "hi-worker", "containerDefinitions": [ { "name": "worker", "image": "tongueroo/hi:ufo-2016-12-04T17-54-23-2d1634d", "cpu": 128, "memoryReservation": 256, "command": [ "bin/worker" ], "environment": [ { "name": "ADMIN_PASSWORD", "value": "secret" } ], "essential": true } ]}$
UFO Template DSLUFO will use a ERB template to generate the task definition for all the processes you need: web, clock, worker, etc.
Allows sharing of code and settings.Let’s see what it looks like.
UFO Task Definition Template DSL$ cat ufo/templates/main.json.erb { "family": "<%= @family %>", "containerDefinitions": [ { "name": "<%= @name %>", "image": "<%= @image %>", "cpu": <%= @cpu %>, <% if @memory %> "memory": <%= @memory %>, <% end %> <% if @memory_reservation %> "memoryReservation": <%= @memory_reservation %>, <% end %> <% if @container_port %> "portMappings": [ { "containerPort": "<%= @container_port %>", "protocol": "tcp" } ], <% end %> "command": <%= @command.to_json %>, <% if @environment %> "environment": <%= @environment.to_json %>, <% end %> <% if @awslogs_group %> "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "<%= @awslogs_group %>", "awslogs-region": "<%= @awslogs_region || 'us-east-1' %>", "awslogs-stream-prefix": "<%= @awslogs_stream_prefix %>" } }, <% end %> "essential": true } ]}$
$ cat ufo/task_definitions.rb common = { image: helper.full_image_name, cpu: 128, memory_reservation: 256, environment: env_file(".env")}
task_definition "hi-web" do source "main" variables(common.dup.deep_merge( family: task_definition_name, name: "web", container_port: helper.dockerfile_port, command: ["bin/web"] ))end
task_definition "hi-worker" do source "main" variables(common.dup.deep_merge( family: task_definition_name, name: "worker", command: ["bin/worker"] ))end
task_definition "hi-clock" do source "main" # will use ufo/templates/main.json.erb variables(common.dup.deep_merge( family: task_definition_name, name: "clock", command: ["bin/clock"] ))end$
UFO Task Definition Output$ cat ufo/output/hi-web.json { "family": "hi-web", "containerDefinitions": [ { "name": "web", "image": "tongueroo/hi:ufo-2016-12-04T17-54-23-2d1634d", "cpu": 128, "memoryReservation": 256, "portMappings": [ { "containerPort": "3000", "protocol": "tcp" } ], "command": [ "bin/web" ], "environment": [ { "name": "ADMIN_PASSWORD", "value": "secret" } ], "essential": true } ]}$
$ cat ufo/output/hi-worker.json { "family": "hi-worker", "containerDefinitions": [ { "name": "worker", "image": "tongueroo/hi:ufo-2016-12-04T17-54-23-2d1634d", "cpu": 128, "memoryReservation": 256, "command": [ "bin/worker" ], "environment": [ { "name": "ADMIN_PASSWORD", "value": "secret" } ], "essential": true } ]}$
ufo ship: whole point of this tool
NOTE: The ECS Cluster and a Container Instance must already exist. I created Container Instances with the lono tool.
1.Build and push docker image
2.Build and register task definition
3.Update the ECS service (create the service automatically if needed)
UFO Demo
ufo ship demo
What Happened Review Summaryufo init - generates the skeleton ufo files.
Mainly ufo/task_definitons.rb and ufo/templates/main.json.erb - most configurations and changes go in here.
ufo/settings.yml - is nice to know about a useful setting in here.
bin/deploy - wrapper script.
In ufo/task_definitions.rb there are some helper methods.
helper.full_image_name - this is the generated docker image name.
helper.dockerfile_port - parsed from the Dockerfile.
env_file - useful helper method to translate a env list to the json formatted list for an ECS Task definition.
Review tasks_definitions.rb Helpers
ufo docker build - only build the docker image
ufo tasks build - only build the task definitions
ufo scale hi-web 1 - scale up and down tasks
Misc ufo commands
THE ENDHacker Noon Blog Post - Ufo — Easily Build Docker Images and Ship Containers to AWS ECS
Setting up CI with UFO, ECS, and CircleCIGitHub - https://github.com/tongueroo/ufo
Slideshare - http://www.slideshare.net/tongueroo/ufo-ship-for-aws-ecs-70885296
[email protected] - feel free to contact me