Australian OpenStack User Group August 2012: Chef for OpenStack

Chef for OpenStack

Matt [email protected]

IRC/Twitter/GitHub: mattraywww.opscode.com/openstack

Deploying and Managing OpenStack is not simple.

Chef makes it easier.

See Node

Application Server

See Nodes

Application Server

Application Database

See Nodes Grow

Application Server

Application Databases

Application Servers

Application Databases

See Nodes Grow

Application Servers

Application Databases

Load Balancer

See Nodes Grow

See Nodes Grow

Application Servers

Application Databases

Load Balancers

See Nodes Grow

Application Servers

Application Database Cache

Load Balancers

Application Databases

Tied together with Config

Application Servers

Application Database Cache

Load Balancers

Application Databases

Infrastructure is a Snowflake

Application Servers

Application Database Cache

Load Balancers

Floating IP?

Application Databases

Evolving Complexity

Load Balancers

Application Servers

NoSQL

Database Slaves

ApplicationCache

Database Cache

Database

Complexity Grows Quickly

DC1

DC3

DC2

http://www.flickr.com/photos/16339684@N00/2681435235/

And it Continues to Evolve

Tell me aboutChef!

Chef is Infrastructure as Code

http://www.flickr.com/photos/louisb/4555295187/

• Programmatically provision and configure

• Treat like any other code base

• Reconstruct business from code repository, data backup, and bare metal resources.

• Chef-Client generates configurations directly on nodes from their run list

• Reduce management complexity through abstraction

• Store the configuration of your programs in version control

http://www.flickr.com/photos/ssoosay/5126146763/

Nodes

Collections of Resources

• Networking

• Files

• Directories

• Symlinks

• Mounts

• Routes

• Users

• Groups

• Tasks

• Packages

• Software

• Services

• Configurations

• Other Stuffhttp://www.flickr.com/photos/stevekeys/3123167585/

Declarative Interface to Resources

• Define policy

• Say what, not how

• Pull not Push

http://www.flickr.com/photos/bixentro/2591838509/

Ruby!

extra_packages = case node['platform'] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } endextra_packages.each do |pkg| package pkg do action :install endend

Recipes and Cookbooks

• Recipes are collections of Resources

• Cookbooks contain recipes, templates, files, custom resources, etc

• Code re-use and modularity

• Hundreds already on Community.opscode.com

http://www.flickr.com/photos/shutterhacks/4474421855/

http://www.flickr.com/photos/kathycsus/2686772625

• IP addresses

• Hostnames

• FQDNs

• Search for nodes with Roles

• Find configuration data

Search

pool_members = search("node","role:webserver”)

template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]"end

Pass Results to Templates

# Set up application listeners here.listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%><% if node["haproxy"]["enable_admin"] -%>listen admin 0.0.0.0:22002 mode http stats uri /<% end -%>

Pass Results to Templates

Jboss App

Memcache

Postgres Slaves

Postgres Master

So when this

NagiosGraphite

Jboss App

Memcache

Postgres Slaves

Postgres Master

NagiosGraphite

Becomes this

Jboss App

Memcache

Postgres Slaves

Postgres Master

NagiosGraphite

Updates can be automatic

NagiosGraphite

Count the resources

Jboss App

Memcache

Postgres Slaves

• Load balancer config

• Nagios host ping

• Nagios host ssh

• Nagios host HTTP

• Nagios host app health

• Graphite CPU

• Graphite Memory

• Graphite Disk

• Graphite SNMP

• Memcache firewall

• Postgres firewall

• Postgres authZ config

• 12+ resource changes for 1 node addition

Build anything

• Simple internal applications

• Complex external applications

• Workstations

• Hadoop clusters

• IaaS infrastructure

• PaaS infrastructure

• SaaS applications

• Storage systems

• You name it

http://www.flickr.com/photos/hyku/245010680/

And manage it simply

http://www.flickr.com/photos/helico/404640681/

• Automatically reconfigure everything

• Linux, Windows, Unixes, BSDs

• Load balancers

• Metrics collection systems

• Monitoring systems

• Cloud migrations become trivial

The Chef Community

• Apache License, Version 2.0

• 800+ Individual contributors

• 150+ Corporate contributors

• HP, Dell, Rackspace, VMware, Joyent, Calxeda, Heroku, SUSE and many more

• 500+ cookbooks

• http://community.opscode.com

ResourcesChef for OpenStack

What's Out There?

Chef for OpenStack: Why

• Community for the automated deployment and management of OpenStack

• Reduce fragmentation and encourage collaboration

• Deploying OpenStack is not "secret sauce"

• Project not a product

• Apache 2 license

Chef for OpenStack: What

• Chef Repository for Deploying OpenStack

• Cookbooks

• Keystone

• Glance

• Nova

• Horizon

• Swift

• Knife OpenStack

Chef for OpenStack: Where

• opscode.com/openstack

• groups.google.com/group/opscode-chef-openstack

• #openstack-chef on irc.freenode.net

• github.com/opscode/openstack-chef-repo

• github.com/opscode-cookbooks/

• keystone, glance, nova, horizon, swift

• github.com/opscode/knife-openstack

Chef for OpenStack: Who

Chef for OpenStack: When

• Essex is working

• KVM

• Ubuntu 12.04

• more to come!

• Folsom about to start

• Milestones for features

• Grizzly and forward

Deploying OpenStack

• Chef ties it all together automatically

• Scaling changes how we deploy

• Interchangeable components

• Configurations shared, supported & documented

• Licensing makes it available to everyone

Rackspace Cloud Builders

• Source of current cookbooks

• github.com/rcbops/chef-cookbooks

• www.rackspace.com/cloud/private/

knife openstack

knife openstack

$ knife openstackAvailable openstack subcommands: (for details, knife SUB-COMMAND --help)

** OPENSTACK COMMANDS **knife openstack flavor list (options)knife openstack image list (options)knife openstack server create (options)knife openstack server delete SERVER [SERVER] (options)knife openstack server list (options)

knife openstack flavor list

$ knife openstack flavor listID Name Virtual CPUs RAM Disk 1 m1.tiny 1 512 MB 0 GB 2 m1.small 1 2048 MB 20 GB 3 m1.medium 2 4096 MB 40 GB 4 m1.large 4 8192 MB 80 GB 5 m1.xlarge 8 16384 MB 160 GB

knife openstack image list

$ knife openstack image listID Name 13 natty-server-cloudimg-amd64 12 natty-server-cloudimg-amd64-kernel 15 oneiric-server-cloudimg-amd64 14 oneiric-server-cloudimg-amd64-kernel

knife openstack server create --node-name ko1 --flavor 1 --image 13 -S trystack

knife openstack server create

$ ssh -i ~/.ssh/trystack.pem [email protected] authenticity of host '8.21.28.24 (8.21.28.24)' can't be established.RSA key fingerprint is 0c:d8:3e:34:d1:de:c4:ee:5f:bc:b5:89:11:0d:73:e0.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added '8.21.28.24' (RSA) to the list of known hosts.Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38-13-virtual x86_64)

* Documentation: https://help.ubuntu.com/

System information as of Thu Feb 16 23:43:29 UTC 2012

System load: 0.08 Processes: 63 Usage of /: 40.8% of 1.35GB Users logged in: 0 Memory usage: 6% IP address for eth0: 8.21.28.24 Swap usage: 0%---------------------------------------------------------------------<snip>Get cloud support with Ubuntu Advantage Cloud Guest http://www.ubuntu.com/business/services/cloud

The programs included with the Ubuntu system are free software;the exact distribution terms for each program are described in theindividual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted byapplicable law.

To run a command as administrator (user "root"), use "sudo <command>".See "man sudo_root" for details.

ubuntu@ko1:~$

Chef for Infrastructure Portability

• knife ec2

• knife rackspace

• knife hp

• knife openstack

• ... and many others

• Documentation

• Hypervisors (LXC, Xen, Hyper-V)

• Databases (PostgreSQL)

• Operating Systems (RHEL)

• HA Configuration

• Quantum

• Cinder

Chef for OpenStack Roadmap

• Cookbooks reusable outside of OpenStack

• Community Events

• Librarian

• Spiceweasel

• pxe_dust

• Crowbar

Chef for OpenStack Ecosystem

Thanks!

Matt [email protected]

IRC/Twitter/GitHub: mattraywww.opscode.com/openstack