Test Driven Infrastructure with Docker, Test Kitchen and Serverspec Yury Tsarev
Apr 13, 2017
Agenda
▸ Goal▸ Test kitchen▸ Docker driver▸ kitchen-puppet provisioner▸ Serverspec verifier▸ Shellmocking▸ Defining puppet type in kitchen▸ Create test-driven infrastructure change▸ Wrap-up
Goal
▸ Infrastructure code should be treated as any other code▸ Apply TDD for puppet▸ Grow regression test suite
Test kitchen
▸ http://kitchen.ci/ ▸ Test orchestrator▸ Originated in Chef community▸ Very pluggable on all levels▸ "Your infrastructure deserves tests too."▸ Book - Test-Driven Infrastructure with Chef
Test kitchen - high level process
1. Create VM/Container (docker driver)2. Run configuration management code there (puppet
provisioner)3. Test with verifier (serverspec)
Main configuration file - .kitchen.yml
▸ Driver: what type of VM/containerization/cloud to use▸ Provisioner: which configuration management tool to apply▸ Verifier: test automation type to verify with▸ Transport: mechanism to upload files into instance under
test
Docker driver
▸ https://github.com/portertech/kitchen-docker▸ kitchen driver to work with docker containers as machines
under testdriver:
name: docker
image: docker-registry.na.intgdc.com:80/gdc:R23
platform: rhel
use_sudo: false
provision_command:
yum clean all && yum makecache
kitchen-puppet provisioner
▸ https://github.com/neillturner/kitchen-puppet▸ Uploads puppet code into instance▸ Runs puppet there (converge)▸ Provides facts customization facility
custom_facts:
docker: 1
hostname: kitchen_test
ec2data_branch: develop # pulp repo branch, critical for gitflow repo
ec2data_gitbranch: develop # git branch
ec2data_freeipa_otp: test
ec2_public_ipv4: 127.0.0.1
ec2_local_ipv4: 127.0.0.1
has_mnt: 0
Serverspec test suite
▸ http://serverspec.org/ - RSpec based framework▸ http://serverspec.org/resource_types.html ▸ Independent of kitchen - can be used standalone▸ We keep test suite together with puppet code under
spec directory to have ability to create consistent PRsdescribe file('/var/log/httpd') do
it { should be_directory }
end
describe docker_image('busybox:latest') do
its(['Architecture']) { should eq 'amd64' }
end
Serverspec verifier: busser or shell
▸ Simple
https://github.com/test-kitchen/busser-serverspec
▸ Advanced
https://github.com/vincentbernat/serverspec-example
Serverspec verifier in GD
▸ Based on shell verifier and serverspec test suite▸ Runs a serverspec test suite against the configured
(converged) instance▸ Reports in shell and junit - jenkins ready
verifier:
name: shell
remote_exec: true
command: |
sudo -s <<SERVERSPEC
export SERVERSPEC_ENV=$EC2DATA_ENVIRONMENT
export SERVERSPEC_BACKEND=exec
serverspec junit=true tag=~skip_in_kitchen check:role:$EC2DATA_TYPE
SERVERSPEC
Transport
▸ https://github.com/coderanger/kitchen-sync▸ Replaced default scp with sftp transport▸ Reduced 1.5m to 5 sec for puppet and test suite upload!
Shellmocking
▸ To bypass external dependencies▸ To bypass docker specific limitations▸ Implementation located in puppet repo under
spec/shellmock▸ Simple ruby script▸ Wraps yum invocation▸ Mock is defined in simple yaml format:
package:
/path/to/executable: contents
Defining puppet type in kitchen
▸ Declare in platforms section of .kitchen.ymlplatforms:
- name: zuul
provisioner:
custom_facts:
ec2data_type: zuul
▸ kitchen converge▸ pray▸ shellmock▸ repeat
Create test-driven infrastructure change
▸ Write serverspec expectation for new code▸ kitchen verify <type>▸ Observe related test is red▸ Write puppet code▸ kitchen converge <type>▸ kitchen verify <type>▸ Observe related test is green▸ Commit the changes and create PR to puppet repo▸ DEMO
Wrap-up: Benefits
▸ Scratch environment▸ Test in isolation▸ Easy to test permutations▸ Resource efficiency▸ Test-first/test-driven approach for infrastructure code▸ Fast feedback - even before a commit▸ Naturally growing regression test suite
Wrap-up: OSS side of things
▸ Multiple open source projects combined▸ We contributed a lot into related kitchen projects and
serverspec▸ Nominated as kitchen-puppet core contributor
Wrap-up: What is next ?
▸ Define all types in kitchen▸ with every next adopted type it will be easier
▸ Create smart way to test only related types for code change, a.k.a puppet dependency tracking
Wrap-up: A note on agnostic approach
▸ As the kitchen driver is agnostic to virtualization solution/cloud
▸ As the kitchen provisioner is agnostic to configuration management solution
▸ As the serverspec is agnostic to way to configure servers at all
▸ We are capable to make brave movements in future!
Wrap-up: Try it!
▸ http://kitchen.ci/ ▸ https://github.com/portertech/kitchen-docker ▸ https://github.com/neillturner/kitchen-puppet ▸ http://serverspec.org/