ReUse Your (Puppet) Modules!

RE-USE YOUR MODULES!Techniques and Approaches to Modules Reusability

Puppet Camp 2010 San FranciscoAlessandro [email protected]

I have a dream

a small one

Cross Vendor

Reusable

Plug & Play

Puppetmodules

Cross Vendor

Reusable

Plug & Play

At the beginning...

class foo {

package { foo: ensure => installed }

service { foo: ensure => running, enable => true, hasstatus => true, require => Package[foo], subscribe => File[foo.conf], }

file { foo.conf: ensure => present, path => “/etc/foo.conf”, mode => 644, owner => root, group => root, source => “puppet:///foo/foo.conf”, }

}

At the beginning

Facing Diversity

class foo {

package { foo: ensure => installed, name => $operatingsystem ? { debian => “food”, default => “foo”, },}

service { foo: ensure => running, name => $operatingsystem ? { debian => “food”, default => “foo”, }, enable => true, hasstatus => true, require => Package[foo], subscribe => File[foo.conf],}

file { foo.conf: ensure => present, path => $operatingsystem ? { debian => “/etc/foo/foo.conf”, default => “/etc/foo.conf”, }, mode => 644,

Facing Diversity

class foo {

package { foo: ensure => installed name => $operatingsystem ? { debian => “food”, default => “foo”, }}

service { foo: ensure => running, name => $operatingsystem ? { debian => “food”, default => “foo”, }, enable => true, hasstatus => $operatingsystem ? { debian => false, default => true, }, pattern => “food”, require => Package[foo], subscribe => File[foo.conf],}

file { foo.conf: ensure => present, path => $operatingsystem ? { debian => “/etc/foo/foo.conf”, default => “/etc/foo.conf”, }, mode => 644,

Facing Diversity

class foo {

package { foo: ensure => installed name => $operatingsystem ? { debian => “food”, ubuntu => “food”, default => “foo”, }}

service { foo: ensure => running, name => $operatingsystem ? { debian => “food”, ubuntu => “food”, default => “foo”, }, enable => true, hasstatus => $operatingsystem ? { debian => false, ubuntu => false, default => true, }, pattern => “food”, require => Package[foo], subscribe => File[foo.conf],}

file { foo.conf: ensure => present, path => $operatingsystem ? { debian => “/etc/foo/foo.conf”, default => “/etc/foo.conf”, }, mode => 644,

Facing Diversity

Centralized settings: foo::paramsQualified variablesOS dedicated classes

Facing Diversity

Centralized settings

class foo::params { $packagename = $operatingsystem ? { debian => “food”, ubuntu => “food” default => "foo", }

$servicename = $operatingsystem ? { debian => “food”, ubuntu => “food”, default => “foo”, },

$processname = $operatingsystem ? { default => "food", }

$hasstatus = $operatingsystem ? { debian => false, ubuntu => false, default => true, }

$configfile = $operatingsystem ? { freebsd => "/usr/local/etc/foo.conf", ubuntu => “/etc/foo/foo.conf", }

}

Facing Diversity

Qualified variables

class foo {

require foo::params

package { "foo": name => "${foo::params::packagename}", ensure => present, }

service { "foo": name => "${foo::params::servicename}", ensure => running, enable => true, hasstatus => "${foo::params::hasstatus}", pattern => "${foo::params::processname}", require => Package["foo"], subscribe => File["foo.conf"], }

file { "foo.conf": path => "${foo::params::configfile}", mode => "${foo::params::configfile_mode}", owner => "${foo::params::configfile_owner}", group => "${foo::params::configfile_group}", ensure => present, require => Package["foo"], notify => Service["foo"], # content => template("foo/foo.conf.erb"), }

Facing Diversity

OS dedicated classes

class foo {

[ ... ]

# Include OS specific subclasses, if necessary # Note that they needn’t to inherit foo

case $operatingsystem { debian: { include foo::debian } ubuntu: { include foo::debian } default: { } }

}

Facing Diversity

Dealing with Users

# foo class needs users’ variables# IE: $foo_server

class foo {

[ ... ]

file { foo.conf: ensure => present, path => $operatingsystem ? { debian => “/etc/foo/foo.conf”, default => “/etc/foo.conf”, }, mode => 644, content => template(“foo/foo.conf.erb”), }}

# foo/templates/foo.conf.erb is something like:

# File Managed by Puppetserver = <%= foo_server %>

# Plug & Play ?

Dealing with Users

Filtering user variables Setting default values

Dealing with Users

Setting default values

class foo::params {

# Full hostname of foo server $server = $foo_server ? { '' => "foo.example42.com", default => "${foo_server}", }

# Foo DB management $db_host = $foo_db_host ? { '' => "localhost", default => "${foo_db_host}", }

$db_user = $foo_db_user ? { '' => "root", default => "${foo_db_user}", }

$db_password = $foo_db_password ? { '' => "", default => "${foo_db_host}", }

Dealing with Users

Setting default values

class foo {

include foo::params

[ ... ]

# ${foo_server} is user variable unfiltered# ${foo::params::server} is user variable filtered

# You may want/need to reassign ${foo_server} value:

${foo_server} = ${foo::params::server}

}

In templates this fails:server = <%= foo::params::server %>

This works (of course):server = <%= foo_server %>

This works (even if variable is not set):server = <%= scope.lookupvar('foo::params::server') %>

Dealing with Users

Adapt and Customize

class foo {

package { foo: ensure => installed name => $operatingsystem ? { debian => “food”, ubuntu => “food”, default => “foo”, }}

service { foo: ensure => running, name => $operatingsystem ? { debian => “food”, ubuntu => “food”, default => “foo”, }, enable => true, hasstatus => $operatingsystem ? { debian => false, ubuntu => false, default => true, }, require => Package[foo], subscribe => File[foo.conf],}

file { foo.conf: ensure => present, path => “/etc/foo.conf”, mode => 644, owner => root, group => root, source => [ “puppet:///foo/foo.conf-$hostname”, “puppet:///foo/foo.conf-$my_role”, “puppet:///foo/foo.conf” ],}

}

Adapt and customize

class foo {

package { foo: ensure => installed name => $operatingsystem ? { debian => “food”, ubuntu => “food”, default => “foo”, }}

service { foo: ensure => running, name => $operatingsystem ? { debian => “food”, ubuntu => “food”, default => “foo”, }, enable => true, hasstatus => $operatingsystem ? { debian => false, ubuntu => false, default => true, }, require => Package[foo], subscribe => File[foo.conf],}

file { foo.conf: ensure => present, path => “/etc/foo.conf”, mode => 644, owner => root, group => root, source => [ “puppet:///foo/foo.conf-$hostname”, “puppet:///foo/foo.conf-$my_role”, “puppet:///foo/foo.conf” ],}

case $my_role { mail: { file { foo.conf-mail: ensure => present, path => “/etc/foo.conf.d/mail”, mode => 644, owner => root, group => root, source => [ “puppet:///foo/foo.conf-mail”, } } default: { }}

}

Adapt and customize

Isolation of diversity Project classes || Project module

Adapt and Customize

Isolation of diversity

# Class foo::example42## You can use your custom project class to modify# the standard behavior of foo module## You don't need to use class inheritance if you# don't override or redefine resources in foo class## You can add custom resources and are free to# decide how to provide the contents of your files:# - Via static sourced files ( source => ) according# to the naming convention you need# - Via custom templates ( content => ) # - Via some kind of infile line modification tools# such as Augeas #class foo::example42 inherits foo { File["foo.conf"] { source => [ "puppet:///foo/foo.conf-$hostname", "puppet:///foo/foo.conf-$role", "puppet:///foo/foo.conf" ], }}

Adapt and Customize

Project Classes ||Project Modules

class foo {

[ ... ]

# Include project specific class if $my_project is set # The extra project class is by default looked in foo module # If $my_project_onmodule == yes it's looked in your project # module

if $my_project { case $my_project_onmodule { yes,true: { include "${my_project}::foo" } default: { include "foo::${my_project}" } } }

}

Adapt and Customize

Control and monitor

class foo {

package { foo: ensure => installed name => $operatingsystem ? { debian => “food”, ubuntu => “food”, default => “foo”, }}

service { foo: ensure => running, name => $operatingsystem ? { debian => “food”, ubuntu => “food”, default => “foo”, }, enable => true, hasstatus => $operatingsystem ? { debian => false, ubuntu => false, default => true, }, pattern => “food”, require => Package[foo], subscribe => File[foo.conf],}

! # Monitoring stuff: munin and nagios! munin::plugin { "foo_processes":! ! ! ensure => present,! }! nagios::service { "foo_${foo_port_real}":! ! check_command => "tcp_port!${foo_port_real}"! }

Control and Monitor

Monitoring abstraction Monitor meta-module

Control and Monitor

Monitoring abstraction

class foo {

[ ... ]

# Include monitor classe if $monitor == yes # Define the monitoring tools to use # with the variables $monitor_tools (can be an array) if $monitor == "yes" { include foo::monitor }

}

Control and Monitor

Monitoring abstraction

class foo::monitor { include foo::params

monitor::port { "foo_${foo::params::protocol}_ ${foo::params::port}": protocol => "${foo::params::protocol}", target => "${foo::params::monitor_target_real}", port => "${foo::params::port}", enable => "${foo::params::monitor_port_enable}", tool => "${monitor_tool}", }

monitor::process { "foo_process": process => "${foo::params::processname}", service => "${foo::params::servicename}", pidfile => "${foo::params::pidfile}", enable => "${foo::params::monitor_process_enable}", tool => "${monitor_tool}", }

[ ... ]

if $my_project { case $my_project_onmodule { yes,true: { include "${my_project}::foo::monitor" } default: { include "foo::monitor::${my_project}" } } }

}

Control and Monitor

Monitor Meta-module

define monitor::process ( $process, $service, $pidfile, $tool, $enable ) {

if ($enable != "false") and ($enable != "no") {

if ($tool =~ /munin/) { # TODO }

if ($tool =~ /collectd/) { # TODO }

if ($tool =~ /monit/) { monitor::process::monit { "$name": pidfile => "$pidfile", process => "$process", service => "$service", } }

if ($tool =~ /nagios/) { monitor::process::nagios { "$name": process => $process, } }

} # End if $enable

}

Control and Monitor

Monitor Meta-module

define monitor::process::monit ( $pidfile='', $process='', $service='' ) {

# Use for Example42 monit module monit::checkpid { "${process}": pidfile => "${pidfile}", startprogram => "/etc/init.d/${service} start", stopprogram => "/etc/init.d/${service} stop", }

# Use for Camptocamp’s monit module (sample) # monit::config { "${process}": # ensure => present, # content => template(“monit/checkprocess.erb”), # To create # }

# Use for Monit recipe on Puppet’s wiki (sample) # monit::package { "${process}": }

}

Control and Monitor

Coherent naming conventionsPredictable behaviors

A name for Everything

Naming conventions

include foo - Installs and runs foo service

# If foo can be client or server include foo::server - Installs foo serverinclude foo::client - Installs foo client

# If foo is on every/many host either client or server:if ($foo_server_local == true) or ($foo_server == "$fqdn") { include puppet::server} else { include puppet::client}

# If foo has to be disable or removed:include foo::absent - Remove fooinclude foo::disable - Disable foo serviceinclude foo::disableboot - Disable foo service but do not check if is running

A name for Everything

Quick cloning & customizationCoherent Modules Infrastructure

Modules Machine

Quick cloning

# Few seds for a script that clones foo moduleexample42_module_clone.sh

# Creates new module from foo template. Then you:# - Edit params.pp# - Add specific classes, defines, types, facts# - Eventually define modules variables and templates## - Everything else is ready out of the box

Modules Machine

Quick customization

# Script that adapts a module for a new projectexample42_project_rename.sh

# Prepares custom classes for new project# foo/manifests/my_project.pp# foo/manifests/monitor/my_project.pp# foo/manifests/backup/my_project.pp## - Use the logic that works for your project # - Choose how to provide your configuration files

Modules Machine

Coherent Infrastructure

# Clone is GOOD# Modules are similar# They share a coherent naming convention# They have confined places where to apply# custom logic# Fixed files for fixes functions# Common approach for the Puppet team members# Reduced risks and better manageability# Quick update to template improvements# Could be more easily manageable/autogenerated# by GUI tools

Modules Machine

Centralized settings: foo::paramsQualified variables

Isolation of diversity Project classes || Project module

Monitoring abstraction Monitor meta-module

Filtering user variables Setting default values

Coherent naming conventionsPredictable behaviors

Quick cloning of foo moduleQuick customization

Cross Vendor Reusable Plug & Play

Modules

MachineA name for

Everything

Dealing

with Users

Control

and MonitorAdapt and

Customize

Facing

Diversity

small dreams turn easier into reality

www.example42.comgithub.com/example42/puppet-foo/

Questions?(Please Loud & Clear, I’ve lost my Babelfish)