Top Banner
Clean manifests with Puppet::Tidy Jasper Lievisse Adriaanse Engineering team, m:tier Puppet Camp 2013 Gent Jan. 31 - Feb. 1, 2013
51
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Clean Manifests with Puppet::Tidy

Clean manifests with Puppet::Tidy

Jasper Lievisse AdriaanseEngineering team, m:tier

Puppet Camp 2013 Gent

Jan. 31 - Feb. 1, 2013

Page 2: Clean Manifests with Puppet::Tidy

Introm:tier

Puppet::TidyFuture ideas

2 of 1

Page 3: Clean Manifests with Puppet::Tidy

Who?

• Somewhat involved in open source...◦ OpenBSD committer since 2006◦ GNOME committer since 2011◦ 35+ public repositories on GitHub

• Involved in m:tier since it’s founding in 2008◦ Started using Puppet in 2009

3 of 1

Page 4: Clean Manifests with Puppet::Tidy

Who?

• Somewhat involved in open source...◦ OpenBSD committer since 2006◦ GNOME committer since 2011◦ 35+ public repositories on GitHub

• Involved in m:tier since it’s founding in 2008◦ Started using Puppet in 2009

3 of 1

Page 5: Clean Manifests with Puppet::Tidy

Who?cont.

Also wrote yasnippet-puppet-mode for Emacsfile<TAB> expands to:

file { "name":

owner => owner,

group => group,

mode => mode,

ensure => ensure,

require => require,

content => content,

source => source;

}

See: http://jasper.la/emacs.html

4 of 1

Page 6: Clean Manifests with Puppet::Tidy

Why Puppet::Tidy?

• puppet-lint

• Easier to understand/expand/fix

5 of 1

Page 7: Clean Manifests with Puppet::Tidy

Introm:tier

Puppet::TidyFuture ideas

6 of 1

Page 8: Clean Manifests with Puppet::Tidy

Puppet in m:tier

• One git repository

• ∼ 6k lines of Puppet code

• Upto “Blue chip” customers

• Three continents

• mtier-puppet

7 of 1

Page 9: Clean Manifests with Puppet::Tidy

Puppet in m:tier

• One git repository

• ∼ 6k lines of Puppet code

• Upto “Blue chip” customers

• Three continents

• mtier-puppet

7 of 1

Page 10: Clean Manifests with Puppet::Tidy

Puppet in m:tier

• One git repository

• ∼ 6k lines of Puppet code

• Upto “Blue chip” customers

• Three continents

• mtier-puppet

7 of 1

Page 11: Clean Manifests with Puppet::Tidy

Puppet in m:tier

• One git repository

• ∼ 6k lines of Puppet code

• Upto “Blue chip” customers

• Three continents

• mtier-puppet

7 of 1

Page 12: Clean Manifests with Puppet::Tidy

Puppet in m:tier

• One git repository

• ∼ 6k lines of Puppet code

• Upto “Blue chip” customers

• Three continents

• mtier-puppet

7 of 1

Page 13: Clean Manifests with Puppet::Tidy

Puppet in m:tiercont.

(Semi-)automated bootstrap of OpenBSD laptops/servers

• LDAP

• Kerberos

• Users

• Packages

• Security updates

• .plocal

8 of 1

Page 14: Clean Manifests with Puppet::Tidy

Puppet in m:tiercont.

For more details:puppetlabs.com/blog/

guest-post-a-puffy-in-the-corporate-aquarium-the-sequel/

9 of 1

Page 15: Clean Manifests with Puppet::Tidy

Introm:tier

Puppet::TidyFuture ideas

10 of 1

Page 16: Clean Manifests with Puppet::Tidy

What is Puppet::Tidy?

• Formal definition:◦ A re-formatter for Puppet manifests, working on syntactic level.

• Informal definition:◦ A glorified bunch of regular expressions which make your Puppet code

look nice.

11 of 1

Page 17: Clean Manifests with Puppet::Tidy

What is Puppet::Tidy?

• Formal definition:◦ A re-formatter for Puppet manifests, working on syntactic level.

• Informal definition:◦ A glorified bunch of regular expressions which make your Puppet code

look nice.

11 of 1

Page 18: Clean Manifests with Puppet::Tidy

How does it work?

• Input is read line-by-line

• One-pass transformation checks

◦ If the line matches criteria, it’s transformed

• Output written to file, or reference passed back

12 of 1

Page 19: Clean Manifests with Puppet::Tidy

How does it work?

• Input is read line-by-line

• One-pass transformation checks

◦ If the line matches criteria, it’s transformed

• Output written to file, or reference passed back

12 of 1

Page 20: Clean Manifests with Puppet::Tidy

How does it work?

• Input is read line-by-line

• One-pass transformation checks◦ If the line matches criteria, it’s transformed

• Output written to file, or reference passed back

12 of 1

Page 21: Clean Manifests with Puppet::Tidy

How does it work?

• Input is read line-by-line

• One-pass transformation checks◦ If the line matches criteria, it’s transformed

• Output written to file, or reference passed back

12 of 1

Page 22: Clean Manifests with Puppet::Tidy

Current checks

• Most common “errors”:◦ expand tabs

◦ comments◦ four digit mode◦ quoting

• attributes• titles• resource reference types• strings

13 of 1

Page 23: Clean Manifests with Puppet::Tidy

Current checks

• Most common “errors”:◦ expand tabs◦ comments

◦ four digit mode◦ quoting

• attributes• titles• resource reference types• strings

13 of 1

Page 24: Clean Manifests with Puppet::Tidy

Current checks

• Most common “errors”:◦ expand tabs◦ comments◦ four digit mode

◦ quoting

• attributes• titles• resource reference types• strings

13 of 1

Page 25: Clean Manifests with Puppet::Tidy

Current checks

• Most common “errors”:◦ expand tabs◦ comments◦ four digit mode◦ quoting

• attributes• titles• resource reference types• strings

13 of 1

Page 26: Clean Manifests with Puppet::Tidy

Current checks

• Most common “errors”:◦ expand tabs◦ comments◦ four digit mode◦ quoting

• attributes

• titles• resource reference types• strings

13 of 1

Page 27: Clean Manifests with Puppet::Tidy

Current checks

• Most common “errors”:◦ expand tabs◦ comments◦ four digit mode◦ quoting

• attributes• titles

• resource reference types• strings

13 of 1

Page 28: Clean Manifests with Puppet::Tidy

Current checks

• Most common “errors”:◦ expand tabs◦ comments◦ four digit mode◦ quoting

• attributes• titles• resource reference types

• strings

13 of 1

Page 29: Clean Manifests with Puppet::Tidy

Current checks

• Most common “errors”:◦ expand tabs◦ comments◦ four digit mode◦ quoting

• attributes• titles• resource reference types• strings

13 of 1

Page 30: Clean Manifests with Puppet::Tidy

Using Puppet::TidyStandalone

#!/usr/bin/perl

use strict;

use Puppet::Tidy;

Puppet::Tidy::puppettidy();

14 of 1

Page 31: Clean Manifests with Puppet::Tidy

Using Puppet::TidyPart of a larger whole

#!/usr/bin/perl

use strict;

use Puppet::Tidy;

my (@output, $source);

$source = << ’EOF’;

Exec[’$reboot’]

EOF

Puppet::Tidy::puppettidy

(source => $source, destination => \@output);

15 of 1

Page 32: Clean Manifests with Puppet::Tidy

Quick demoinput

Exec[’$reboot’] // What is it doing here?

file {

"/tmp/blah":

mode => 644

}

package { ’$openssh’:

ensure => present

}

16 of 1

Page 33: Clean Manifests with Puppet::Tidy

Quick demooutput

Exec[$reboot] # What is it doing here?

file {

’/tmp/blah’:

mode => ’0644’

}

package { "$openssh":

ensure => present;

}

17 of 1

Page 34: Clean Manifests with Puppet::Tidy

Installing Puppet::Tidy

• CPAN

◦ $ cpan Puppet::Tidy

• OpenBSD

◦ $ pkg add p5-Puppet-Tidy

18 of 1

Page 35: Clean Manifests with Puppet::Tidy

Installing Puppet::Tidy

• CPAN◦ $ cpan Puppet::Tidy

• OpenBSD

◦ $ pkg add p5-Puppet-Tidy

18 of 1

Page 36: Clean Manifests with Puppet::Tidy

Installing Puppet::Tidy

• CPAN◦ $ cpan Puppet::Tidy

• OpenBSD

◦ $ pkg add p5-Puppet-Tidy

18 of 1

Page 37: Clean Manifests with Puppet::Tidy

Installing Puppet::Tidy

• CPAN◦ $ cpan Puppet::Tidy

• OpenBSD◦ $ pkg add p5-Puppet-Tidy

18 of 1

Page 38: Clean Manifests with Puppet::Tidy

Introm:tier

Puppet::TidyFuture ideas

19 of 1

Page 39: Clean Manifests with Puppet::Tidy

Basic features

• Define checks to run

• Moar checks!

• Attribute alignment

• Output validation

20 of 1

Page 40: Clean Manifests with Puppet::Tidy

Basic features

• Define checks to run

• Moar checks!

• Attribute alignment

• Output validation

20 of 1

Page 41: Clean Manifests with Puppet::Tidy

Basic features

• Define checks to run

• Moar checks!

• Attribute alignment

• Output validation

20 of 1

Page 42: Clean Manifests with Puppet::Tidy

Basic features

• Define checks to run

• Moar checks!

• Attribute alignment

• Output validation

20 of 1

Page 43: Clean Manifests with Puppet::Tidy

Semantic knowledge

• Puppet::Tidy just works on syntactic level

• It should be smarter and know about blocks

◦ find the blocks◦ group the blocks◦ and in the file bind them

21 of 1

Page 44: Clean Manifests with Puppet::Tidy

Semantic knowledge

• Puppet::Tidy just works on syntactic level

• It should be smarter and know about blocks

◦ find the blocks◦ group the blocks◦ and in the file bind them

21 of 1

Page 45: Clean Manifests with Puppet::Tidy

Semantic knowledge

• Puppet::Tidy just works on syntactic level

• It should be smarter and know about blocks◦ find the blocks

◦ group the blocks◦ and in the file bind them

21 of 1

Page 46: Clean Manifests with Puppet::Tidy

Semantic knowledge

• Puppet::Tidy just works on syntactic level

• It should be smarter and know about blocks◦ find the blocks◦ group the blocks

◦ and in the file bind them

21 of 1

Page 47: Clean Manifests with Puppet::Tidy

Semantic knowledge

• Puppet::Tidy just works on syntactic level

• It should be smarter and know about blocks◦ find the blocks◦ group the blocks◦ and in the file bind them

21 of 1

Page 48: Clean Manifests with Puppet::Tidy

Real parser

• Currently works on a line-by-line basis, but could use a real parser

• Anyone got experience with MARPA?

22 of 1

Page 49: Clean Manifests with Puppet::Tidy

Real parser

• Currently works on a line-by-line basis, but could use a real parser

• Anyone got experience with MARPA?

22 of 1

Page 50: Clean Manifests with Puppet::Tidy

Questions || suggestions?

23 of 1

Page 51: Clean Manifests with Puppet::Tidy

Thank you!and thank to my employer m:tier for sponsoring the developmentof Puppet::Tidy.

mail [email protected]

www jasper.la and www.mtier.org

twitter jasper lagithub jasperla

CPAN search.cpan.org/~jasper

24 of 1