Infrastructure Automation With Opscode Chef Presentation

Infrastructure Automation with Opscode Chef

http://[email protected]#opschef

Tuesday, June 14, 2011

Who are we?

Joshua Timberman Adam Jacob Christopher Brown Aaron Peterson Seth Chisamore Matt Ray

Tuesday, June 14, 2011

Who are you?

System administrators? Developers? Business People?

http://www.flickr.com/photos/timyates/2854357446/sizes/l/

Tuesday, June 14, 2011

Hint, consultants, youre Business people too.

What are we talking about?

http://www.flickr.com/photos/peterkaminski/2174679908/

Tuesday, June 14, 2011

Managing infrastructure in the Cloud. With Chef, hopefully.

Agenda

Hows and Whys Live Demo! Getting Started with Chef Anatomy of a Chef Run Managing Cloud Infrastructure Data Driven Shareable Cookbooks

http://www.flickr.com/photos/koalazymonkey/3590953001/

Tuesday, June 14, 2011

Hows and whys of managing infrastructure with Chef.Were running a live demo!Well walk through the things required to get started with Chef.We will look at the anatomy of a Chef run in detail.Since weve launched a cloud infrastructure, well want to know how we manage it.Well talk about our data driven sharable cookbooks.

Infrastructure as Code

Tuesday, June 14, 2011

The goal is fully automated infrastructure. In the cloud, anywhere. We get there with Infrastructure as Code.

A technical domain revolving around building and managing infrastructure programmatically

Tuesday, June 14, 2011

Enable the reconstruction of the business from

nothing but a source code repository, an application

data backup, and bare metal resources.

Tuesday, June 14, 2011

Configuration Management

Tuesday, June 14, 2011

Keep track of all the steps required to take bare metal systems to doing their job in the infrastructure.It is all about the policy.And this needs to be available as a service in your infrastructure.

System Integration

http://www.flickr.com/photos/opalsson/3773629074/

Tuesday, June 14, 2011

Taking all the systems that have been configured to do their job, and make them work together to actually run the infrastructure.

Tuesday, June 14, 2011

Introducing Chef.Maybe youve already met!Stephen Nelson-Smith has a great way to introducing Chef, so with apologies to him, Im going to reuse his descriptions.

The Chef Framework

With thanks (and apologies) to Stephen Nelson-SmithTuesday, June 14, 2011

Chef provides a framework for fully automating infrastructure, and has some important design principles.

The Chef Framework

Reasonability Flexibility Library & Primitives TIMTOWTDI

Tuesday, June 14, 2011

Chef makes it easy to reason about your infrastructure, at scale. The declarative Ruby configuration language is easy to read, and the predictable ordering makes it easy to understand whats going on.Chef is flexible, and designed to allow you to build infrastructure using a sane set of libraries and primitives.Just like Perl doesnt tell programmers how to program, Chef doesnt tell sysadmins how to manage infrastructure.

The Chef Tool(s)

With thanks (and apologies) to Stephen Nelson-SmithTuesday, June 14, 2011

Since Chef is a framework with libraries and primitives for building and managing infrastructure, it only makes sense that it comes with tools written for that purpose.

The Chef Tool(s)

ohai chef-client knife shef

Tuesday, June 14, 2011

Ohai profiles the system to gather data about nodes and emits that data as JSON.Chef client runs on your nodes to configure them.Knife is used to access the API.Shef is an interactive console debugger.

The Chef API

With thanks (and apologies) to Stephen Nelson-Smith

Tuesday, June 14, 2011

The Chef API provides a client/server service for configuration management in your infrastructure.

The Chef API

RSA key authentication w/ Signed Headers RESTful API w/ JSON Search Service Derivative Services

Tuesday, June 14, 2011

The API itself is RESTful with JSON responses.Part of the API is a dynamic search service which can be queried to provide rich data about the objects stored on the server.Because it is flexible and built as a service, it is easy to build derivative services on top, including integration with other tools and services.

The Chef Community

With thanks (and apologies) to Stephen Nelson-SmithTuesday, June 14, 2011

As an Open Source project, the Chef community is critical.

The Chef Community

Apache License, Version 2.0 360+ Individual contributors 70+ Corporate contributors Dell, Rackspace,VMware, RightScale,

Heroku, and more

http://community.opscode.com 240+ cookbooks

Tuesday, June 14, 2011

Community is important.http://apache.org/licenses/LICENSE-2.0.htmlhttp://www.opscode.com/blog/2009/08/11/why-we-chose-the-apache-license/http://wiki.opscode.com/display/chef/How+to+Contributehttp://wiki.opscode.com/display/chef/Approved+Contributors

Chef Enables Infrastructure as Code

Resources Recipes Roles Source Code

package "haproxy" do action :installend

template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode 0644 notifies :restart, "service[haproxy]"end

service "haproxy" do supports :restart => true action [:enable, :start]end

Tuesday, June 14, 2011

Declare system configuration as idempotent resources.Put resources together in recipes.Assign recipes to systems through roles.Track it all like source code.

Chef Resources

Have a type. Have a name. Have parameters. Take action to put the resource

in the declared state.

Can send notifications to other resources.

package "haproxy" do action :installend

template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode 0644 notifies :restart, "service[haproxy]"end

service "haproxy" do supports :restart => true action [:enable, :start]end

Tuesday, June 14, 2011

Resources take action through Providers

Tuesday, June 14, 2011

Providers know how to actually configure the resources to be in the declared state

package haproxy { yum install haproxyapt-get install haproxypacman sync haproxypkg_add -r haproxy

Chef Providers

Tuesday, June 14, 2011

The haproxy package resource may run any number of OS commands, depending on the nodes platform.

Recipes are collections of Resources

Tuesday, June 14, 2011

Chef Recipes

Recipes are evaluated for resources in the order they appear.

Each resource object is added to the Resource Collection.

package "haproxy" do action :installend

template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode 0644 notifies :restart, "service[haproxy]"end

service "haproxy" do supports :restart => true action [:enable, :start]end

Tuesday, June 14, 2011

Chef Recipes

Recipes can include other recipes.

Included recipes are processed in order.

include_recipe "apache2"include_recipe "apache2::mod_rewrite"include_recipe "apache2::mod_deflate"include_recipe "apache2::mod_headers"include_recipe "apache2::mod_php5"

Tuesday, June 14, 2011

Just like recipes themselves are processed in order, the recipes included are processed in order, so when you include a recipe, all its resources are added to the resource collection, then Chef continues to the next.

Chef Recipes

Extend recipes with Ruby.

Iterate over an array of package names to install.

%w{ php5 php5-dev php5-cgi }.each do |pkg|

package pkg do action :install end

end

Tuesday, June 14, 2011

Chef Recipes

Good: Drop off a dynamic template.

Better: Discover data through search.

pool_members = search("node", "role:mediawiki")

template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members notifies :restart, "service[haproxy]"end

template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode 0644 notifies :restart, "service[haproxy]"end

Tuesday, June 14, 2011

Chef Roles

Roles describe nodes. Roles have a run list. Roles can have attributes.

name "mediawiki"description "mediawiki app server"run_list( "recipe[mysql::client]", "recipe[application]", "recipe[mediawiki::status]")

name "mediawiki_load_balancer"description "mediawiki load balancer"run_list( "recipe[haproxy::app_lb]")override_attributes( "haproxy" => { "app_server_role" => "mediawiki" })

Tuesday, June 14, 2011

Track it like source code...

% git logcommit d640a8c6b370134d7043991894107d806595cc35Author: jtimberman

Import nagios version 1.0.0

commit c40c818498710e78cf73c7f71e722e971fa574e7Author: jtimberman

installation and usage instruction docs

commit 99d0efb024314de17888f6b359c14414fda7bb91Author: jtimberman

Import haproxy version 1.0.1

commit c89d0975ad3f4b152426df219fee0bfb8eafb7e4Author: jtimberman

add mediawiki cookbook

commit 89c0545cc03b9be26f1db246c9ba4ce9d58a6700Author: jtimberman

multiple environments in data bag for mediawiki

Tuesday, June 14, 2011

LIVE DEMO!!!

git clone git://github.com/opscode/velocity2011-chef-repo

Tuesday, June 14, 2011