Testing for infrastructure as code for AnsibleFest 2016

© Mike Caspar, 2016Testing for Infrastructure as Code by Mike Casparis licenced under an Attribution-ShareAlike 4.0 International licence

Testing for Infrastructure as Code

A Beginner's PatternPresented by: Mike Caspar

Incremental Infrastructure Coach


The Audience?


Caspar Computer Services Inc.-- Innovative Solutions since 1984 --

Caspar.NET (5 cities in Canada / 10 years)


Why am I here?


How do we start?Very basic understanding of Ansible Inventory & Groups

A sample environment

A starting pattern (The Test/Maintain Loop)

Moving machines between environments

Integrating Testing into the process (TDD for Infrastructure)

Q & A


Physical Servers Virtual Servers

Ansible works with both.


Applications…

Not servers.


Physical and Virtual Infrastructure

Production

Development

Staging (sort of)


Inventory - Folders by Application Name

WALDO and GROVER are Application names…

Why is this important ?


Inventory - Host Location and Types

City

Env.


Application - Group Variables

---# Playbook: GROVER_maintain_proxy.yml# This playbook Maintains the GROVER Application

- name: ensure hosts are setup correctly on proxy servers in yyz hosts: GROVER_yyz_proxy_staging, GROVER_yyz_proxy_dev, GROVER_yyz_proxy_prod remote_user: root


Tests can be attached and versioned alongwith a specific version of Infrastructure

(think... testing & rollbacks)

Source Control…. Why ?


Host Names not IP Addresses !(virtual machines)


group_vars matching basics

Playbook hosts + Inventory + group_vars + --limit


group_vars matching basics

Playbook hosts + Inventory + group_vars + --limit

- name: ensure hosts are setup correctly on proxy servers in yyz

hosts: GROVER_yyz_proxy_staging, GROVER_yyz_proxy_dev, GROVER_yyz_proxy_prod

remote_user: root

Playbook

Inventory

group_vars[GROVER_yyz_proxy_prod]yyzproxy1.caspar.com

[GROVER_yyz_webservers_prod]web1.caspar.com


The Test/Maintain Loop

_test(govern)

_maintain (dev)

_setup(injected once)


The continual loop

( Dev/Test Continuous Loop)

APP_Maintain_serverType.ymlAPP_Test_serverType.ymlAPP_Maintain_serverType.ymlAPP_Test_serverType.yml



Adding a new Application (in dev/staging first)

APP_maintain_serverType.ymlAPP_test_serverType.ymlAPP_setup_serverType.yml (injected into loop only once).APP_test_serverType.yml


_setup

( _setup injected only once )

_test(govern)

_maintain (dev)


---# Playbook: GROVER_maintain_proxy.yml# This playbook Maintains the GROVER Application

- name: ensure hosts are setup correctly on proxy servers in yyz hosts: GROVER_yyz_proxy_staging, GROVER_yyz_proxy_dev, GROVER_yyz_proxy_prod remote_user: root

roles:

- { role: GROVER-common } - { role: GROVER-users } - { role: GROBER-firewalls } tags: ['security'] } - { role: COMMON-newrelic, state: present, tags: ['monitoring'] } - { role: COMMON_puppet_agent, state: absent }

- name: ensure hosts are setup correctly on proxy servers in yul hosts: GROVER_yul_proxy_staging, GROVER_yul_proxy_dev, GROVER_yul_proxy_prod remote_user: root

roles:

- { role: GROVER-common } - { role: GROVER-users } - { role: GROBER-firewalls } tags: ['security'] } - { role: COMMON-newrelic, state: present, tags: ['monitoring'] } - { role: COMMON_puppet_agent, state: absent }

Notice the Application Name


We decide when a machine is in Dev/Staging/Production(not physical location)

Production

Development

Staging (sort of)

To move a machine from Staging to Production….

Move it to another Inventory and run APP_maintain_


ansible-playbook -i Inventory/GROVER/yyz/dev/GROVER_test_proxy.yml ansible-playbook -i Inventory/GROVER/yyz/staging/GROVER_test_proxy.yml ansible-playbook -i Inventory/GROVER/yyz/prod/GROVER_test_proxy.yml

ansible-playbook -i Inventory/GROVER/yyz/dev/GROVER_test_proxy.yml --limit a.host.comansible-playbook -i Inventory/GROVER/yyz/staging/GROVER_test_proxy.yml --limit a.host.com ansible-playbook -i Inventory/GROVER/yyz/prod/GROVER_test_proxy.yml --limit a.host.com

ansible-playbook -i Inventory/GROVER/yyz/dev/GROVER_maintain_proxy.ymlansible-playbook -i Inventory/GROVER/yyz/staging/GROVER_maintain_proxy.ymlansible-playbook -i Inventory/GROVER/yyz/prod/GROVER_maintain_proxy.yml

Example commands

White - DevBlue - QA/Test

Orange - Governance/Ops


Be careful of role defaults

Empty Strings

Length of group_vars tag


Why test_ instead of --check?


TDD - Test Driven Development(learned from the Software Community)

THINK !

Write a Failing Test

Refactor

Make it Pass

http://www.extremeprogramming.org/rules/testfirst.html

http://www.extremeprogramming.org/rules/testfirst.html


Why is this a useful way to start?

● New apps are not re-deployed, the machines simply move inventory location

● tests have already run many times

● Testing and Governance work can be done in parallel to development tasks

● Development teams know right away if they broke something before and/or after !

● Ability to use TDD software concepts for hardware

● Infrastructure can contain a version with matching tests


T.D.D. concepts for Infrastructure

To truly do Incremental Infrastructure delivery, we must have an automated way to know that we haven’t broken something else in the system when we make changes.

The key is finding a method to allow constant evolution of our code base (infrastructure).

We do not need to reinvent an approach. Test Driven Development concepts have proven effective in incremental software delivery and can be re-used effectively for infrastructure as code..

Mike Caspar, 2016


Git Repository for Sample Project

Sample project file(s) available here…

https://github.com/MikeCaspar/ansibleFest2016SFO

Mike Caspar, 2016

https://github.com/MikeCaspar/ansibleFest2016SFO


Mike Caspar

Incremental Infrastructure CoachEnterprise agile Coach

email: [email protected]: @mikecasparblog: http://mike-caspar.blogspot.com

Testing for infrastructure as code for AnsibleFest 2016

Presentations & Public Speaking