Top Banner
Chef Cookbook Design Patterns Eric Krupnik Software Engineer Copyright Clearance Center
33

Chef Cookbook Design Patterns

Jan 07, 2017

Download

Technology

Eric Krupnik
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Chef Cookbook Design Patterns

Chef Cookbook Design Patterns

Eric KrupnikSoftware EngineerCopyright Clearance Center

Page 2: Chef Cookbook Design Patterns

05/02/20232

Who Am I?

• Software Engineer

• Platform Engineering at CCC

• Chef user since May, 2013

[email protected]

[email protected]

github.com/ekrupnik

Page 3: Chef Cookbook Design Patterns

05/02/20233

Who Are You?

• Who has heard of Chef?

• Who uses Chef?– Other configuration management tools?

Page 4: Chef Cookbook Design Patterns

05/02/20234

Agenda

• Introduction

• Chef overview

• Cookbook types

• Data bags vs attributes

• Run lists and cookbook management

• Q and A

Page 5: Chef Cookbook Design Patterns

05/02/20235

INTRODUCTION

Page 6: Chef Cookbook Design Patterns

05/02/20236

Issues To Address

• Large volume of applications– Number of apps

– Many environments

– Too much to handle manually!

• Error prone configuration during deployments

Page 7: Chef Cookbook Design Patterns

05/02/20237

Sheer Volume

• Many environments (columns)

• Many applications (rows)

• This leads to error prone deployments and inevitable configuration issues

• As of May, 2015 more than 180 deployable components over 6 major environments!

Page 8: Chef Cookbook Design Patterns

05/02/20238

CHEF OVERVIEW

Page 9: Chef Cookbook Design Patterns

05/02/20239

What is Chef? Infrastructure as Code?

• Configuration management tool

• Infrastructure as code– Executable text

• Don’t write MS Word documents anymore

– Version control• Track how infrastructure changes over time• Shareable

– Idempotent• Write code to define how a machine should look• Run Chef often to prevent configuration drift

– Testable

Page 10: Chef Cookbook Design Patterns

05/02/202310

Chef Terms

• A cookbook is the versioned unit of distribution– We will cover cookbook types later

– Contains 1+ recipe(s)

• A recipe is where a user defines a set of resources Chef manages

• Recipe should manage a group of related resources

• A resource is “something” Chef manages and configures– Package, directory, file, service, user

– Can be configured with attributes

– Can create custom resources

Page 11: Chef Cookbook Design Patterns

05/02/202311

Chef Resource Example

directory "/tmp/folder" do

owner 'root'

group 'root'

.

.

mode node['folder_priv']

action :create

end

Declaration of resource

Hard coded attribute

Hard coded attribute

.

.

Node attribute

Action to perform (create, delete)

end

Page 12: Chef Cookbook Design Patterns

05/02/202312

COOKBOOK TYPES

Page 13: Chef Cookbook Design Patterns

05/02/202313

Cookbook Types

• Application cookbook

• Library cookbook

• Wrapper cookbook– Run differently configured instances of application

– Won’t cover this today because it is rare

• Community cookbook– The community maintains many cookbooks you can use

– Won’t cover this today

Page 14: Chef Cookbook Design Patterns

05/02/202314

Application Cookbook

• One recipe per deployable component

• Define attributes and configurations needed for application– Attributes like application name

– Property file(s)

• Should be fairly lightweight

• Should depend on one or more library cookbooks

• Suggested naming conventions:– Cookbook – for the application it manages

– Recipes – for the individual deployable component

Page 15: Chef Cookbook Design Patterns

05/02/202315

Application Cookbook

cookbooks/myapp /attributes ui.rb rest_svc.rb /files/test ui_test.rb rest_svc_test.rb /recipes ui.rb rest_svc.rb /templates ui_properties.erb rest_svc_properties.erb

metadata.rb Berksfile .kitchen.yml

Page 16: Chef Cookbook Design Patterns

05/02/202316

Library Cookbook

• Defines a standard way to deploy an artifact

• Consumed by application cookbooks

• Can link multiple library cookbooks together

• Keeps application cookbooks DRY

• Contains any custom resources– Act on attributes defined by application cookbook

– Has defaults, can be overwritten by application cookbook

Page 17: Chef Cookbook Design Patterns

05/02/202317

Custom Resources

• This is called an LWRP (Lightweight Resource Provider)– Heavy weight resource providers also exist

• These should be the biggest worker in your cookbooks. It’s reusable, configurable, and defines how an application should be deployed.

• Two primary files– Resource: defines accepted attributes/configuration

– Provider: defines the “how” and does the work

– Each named after the resource you are defining

Page 18: Chef Cookbook Design Patterns

05/02/202318

Custom Resources (Resource)

lib_cookbook/resources/webapp.rb

actions :install, :delete

attribute :app_name, :kind_of => String, :required => trueattribute :port, :kind_of => String, :required => true

.

.

attribute :property_file_tempalte, :kind_of => String, :required => false

default_action :install

Page 19: Chef Cookbook Design Patterns

05/02/202319

Custom Resources (Provider)lib_cookbook/provider/webapp.rb

action :install do

app = new_resource.app_name java_version = Helpers.get_java_version(app)

java_install java_version do java_version java_version install_dir “/ccc/java” end

remote_file “#{app}” do source Helpers.get_artifactory_url(app) owner “root” end

template “/ccc/apps/#{app}/config/#{app}.properties” do source property_file_template mode 0644 variables( :data_bag_values => Helpers.get_data_bag_values(app) ) end

end

Page 20: Chef Cookbook Design Patterns

05/02/202320

DATA BAGS VS ATTRIBUTES

Page 21: Chef Cookbook Design Patterns

05/02/202321

Defining Configurations

• So far, we have covered setting attributes in application cookbooks– Pros:

• Forced standard in all environments• Versioned as part of a cookbook

– Cons:• Can’t change attribute values between environments• Hard to change “quickly and easily”

• Data bags are the answer!– Data bag contains 1 data bag item (JSON file) per environment

– 1:1 data bag to cookbook (same name even!)

– Live configuration data including application version

Page 22: Chef Cookbook Design Patterns

05/02/202322

Data Bag Example

data_bags/myapp/DEV.json

{ “id”: “DEV”, “myapp.welcomePage.title”: “Welcome to our app!”, “myapp.error.message”: “This is DEV – what did you expect! Of course it crashed!”, “myapp.error.email.recipient.list”: “[email protected], [email protected]” }

data_bags/myapp/QA.json

data_bags/myapp/UAT.json

data_bags/myapp/PRD.json

{ “id”: “PRD”, “myapp.welcomePage.title”: “Welcome to our app!”, “myapp.error.message”: “We’re sorry something went wrong.”, “myapp.error.email.recipient.list”: “[email protected]” }

Page 23: Chef Cookbook Design Patterns

05/02/202323

Data Bags in Action

• Remember this from the custom resource? template “/ccc/apps/#{app}/config/#{app}.properties” do source property_file_template variables( :data_bag_values => Helpers.get_data_bag_values(app) ) end

• Since the templates are erb files which allow deploy time variable replacement, we can use data bag values to generate a properties file template for the appropriate environment as we deploy.

Page 24: Chef Cookbook Design Patterns

05/02/202324

Data Bags in Action

• Properties file template: cookbooks/myapp/templates/myapp.properties.erb

myapp.welcomePage.title=<%= @data_bag_values[‘myapp.welcomPage.title’] %> myapp.error.message=<%= @data_bag_values[‘myapp.error.message’] %> myapp.error.email.recipient.list=<%= @data_bag_values[‘myapp.error.email.recipient.list’] %>

• Generated at deploy time by our custom resource we get: DEV: myapp.welcomePage.title=Welcome to our app! myapp.error.message=This is DEV – what did you expect! Of course it crashed! [email protected], [email protected]

PRD: myapp.welcomePage.title=Welcome to our app! myapp.error.message=We’re sorry something went wrong. [email protected]

Page 25: Chef Cookbook Design Patterns

05/02/202325

RUN LISTS AND COOKBOOK MANAGEMENT

Page 26: Chef Cookbook Design Patterns

05/02/202326

Run Lists

• A run list defines which recipes are needed to run during a Chef run.

• Run lists are stored on the Chef Server, and are specific to a node.

• Since library cookbooks only get called by application cookbooks, no library cookbook recipes should go into the run list

• Run lists are most commonly defined in either:– Chef role

– Chef environment file

Page 27: Chef Cookbook Design Patterns

05/02/202327

Roles vs Environment Files

• Both are unversioned, but roles tend to be more dangerous due to more global effect

• Both CAN define global or environment-wide attributes

• Environment files contain cookbook version to use

• Simple to keep cookbook versions and run lists in same place

• We do not use roles for these reasons!

Page 28: Chef Cookbook Design Patterns

05/02/202328

Environment Files Example

environments/DEV.rb name “DEV” cookbook_versions({ “myapp” => “= 3.1.0”, “newapp” => “= 1.4.0”, . . )} run_lists({ “ui-server” => “recipe[my_app::ui]”, “svc-server” => “recipe[my_app::rest_svc], recipe[newapp::rest_svc]” )}

environments/QA.rbenvironments/UAT.rb

environments/PRD.rb name “PRD” cookbook_versions({ “myapp” => “= 3.1.0”, . . )} run_lists({ “ui-server” => “recipe[my_app::ui]”, “svc-server” => “recipe[my_app::rest_svc]” )}

Page 29: Chef Cookbook Design Patterns

05/02/202329

Cookbook Dependencies

• Cookbooks can depend on other cookbooks– Application cookbook depends on library cookbook(s)

– Library cookbook depends on other library cookbook(s)

• Each cookbook has a metadata.rb file– Defines info on cookbook

– Defines cookbook dependencies

– Doesn’t matter if library or application cookbook

• There are tools like Berkshelf and Librarian– Berkshelf is now included in Chef-DK!

Page 30: Chef Cookbook Design Patterns

05/02/202330

Cookbook metadata.rb Example

name “myapp”

maintainer “Copyright Clearance Center”

maintainer_email “[email protected]

description “App cookbook to install myapp”

version “3.1.0”

depends “library_cookbook_1”, “=5.0.0”

depends “library_cookbook_2”

depends “library_cookbook_3”, “~> 1.0”

depends “community_cookbook”

Page 31: Chef Cookbook Design Patterns

05/02/202331

Summary

• Recipes should be light weight and very focused

• Use library cookbooks with custom resources

• Library cookbooks can use other library cookbooks

• Use a mix between attributes and data bags– Standard configurations in attributes

– Environment specific configurations in data bags

• Use environment files for cookbook versions and run lists

Page 32: Chef Cookbook Design Patterns

05/02/202332

QUESTIONS?

Page 33: Chef Cookbook Design Patterns

Thank you!Eric Krupnik & CCC Platform Engineering team