Infrastructure Automation at Scale With Puppet and Friends Nick Jones, Senior Cloud Systems Engineer 11 th September, 2014
Infrastructure Automation at Scale
With Puppet and Friends
Nick Jones, Senior Cloud Systems Engineer
11th September, 2014
Introduction
Scalability
“Scalability is the ability of a system, network, or process to handle a growing amount of work in a capable manner or its ability to be enlarged to accommodate that growth.”
Scale-ability
• Repeatability
• Flexibility
• Diversity
• Transparency
• Automate!
• Configuration management and automation
• Programmatically define server and service configuration
• Relationships between services
• Driven by data
• Describe and document infrastructure
• Scalable
Puppet
Puppet Community
Neutron = 8422 LOC
Nova = 6678 LOC
Glance = 3336 LOC
Cinder = 6379 LOC
Horizon = 1681 LOC
Keystone = 4020 LOC
Ceilometer = 3710 LOC
~ 5000 CPUs, 1TB RAM
} 2478 LOC to deploy
OpenStack
Open Source
Puppet Code
# Install the MIBS meta-package package { 'snmp-mibs-downloader': ensure => latest, }
file { 'snmp.conf': require => Package['snmp-mibs-downloader'], path => '/etc/snmp/snmp.conf', content => 'com2sec local localhost public', }
Puppet Deployment Patterns
• A role is a high-level (business) description of a node’s function
• Roles decompose into profiles that include configuration specifics
• Profiles include various modules and take care of declaring the necessary classes and resources
The Foreman
• Lifecycle management• Physical and virtual• External node classification
(ENC) for Puppet
The Foreman - Architecture
Foreman and Puppet
Puppet
Foreman
Node
ProvisionData
Configuration
Foreman Dashboard
Developers, developers, developers, developers…
Development Process• Git for distributed version
control• Work on ‘feature branches’
locally• Push to local Git repo,
mirrored onto GitHub• GitHub’s “social” aspects for
code review• Merge into ‘master’ branch
on team consensus
Development Tools
• Packer• Virtual machine image creation
• Template stored in Foreman
• Automatically creates Vagrant boxes
• Vagrant• Define virtual environments• Puppet provisioner• Portable