DevOps for Drupal: Why We Cook With Chef
Post on 15-Jan-2015
302 Views
Preview:
DESCRIPTION
Transcript
DevOps for Drupal: Why We Cook with Chef
Wednesday May 22nd DevOps Track
Beginner
Overview: Purpose
Why are we all here?
• Interested in DevOps practices
• Interested in automation tools like Chef
• Interested in using them with Drupal
+
Overview: Sharing
How are we going to do this?
• Team style presentation and discussion with views from both sides - Development and Operations
• Share our experiences and examples... we will be keeping it real
• We are NOT doing a tool comparison
+
Overview: Learn
What do we hope you will learn?
• Why DevOps matters
• Why automation is a must
• Why you can use Chef to help yourselves, make life easier, and save time
+
For more info on Chef, check out fellow DevOps track session:
The Joy of Cooking - Whip Up a Drupal Environment with Chef
• Thursday May 23rd at 2:15pm • OR 203 • Presented by Opscode's Nathen Harvey
Overview: More on Chef +
Who We Are
A software development shop... • specializing in custom Drupal development,
systems integration, mobile, DevOps, and 24x7 support
• based in Chicago with 30+ team members worldwide
...and yes we are hiring! Come talk to us!
+
Who We Are Playing the part of:
Moderator and Product Owner
@ Promet Source: Director of Products jay@promethost.com
+
Jay Uhlinger
Who We Are Playing the part of:
Development
@ Promet Source: Solutions Architect will@promethost.com
drupal.org: wamilton twitter/hub: @winmillwill
+
Will Milton
Who We Are Playing the part of:
Operations
@ Promet Source: Systems Administrator greg@promethost.com
+
Greg Palmier
Audience Quiz Time
How many are Developers? (a Will)
+
CC Image courtesy of Pedro Lozano on Flickr
Audience Quiz Time
How many are Sysadmins? (a Greg)
+
CC Image courtesy of Sharyn Morrow on Flickr
Audience Quiz Time
How many are both? (changing Prod servers)
+
CC Image courtesy of Arthur Caranta on Flickr
Audience Quiz Time
How many are other? (the Product Manager)
+
Audience Quiz Time
How many are using automated configuration management tools and processes now in your everyday work with Drupal?
How many are using Chef?
+
DevOps: What It Is
Key Culture • Foster a collaborative working relationship between
Development and Operations
Key Concept • Infrastructure as code
Key Results • Higher deployment rates • Better reliability, stability, and resilience of Production
+
DevOps: 3 Key Patterns
1. Make environments available early in the Development process
2. Shorten and amplify feedback loops 3. Create reusable deployment procedures
+
DevOps: What It Isn't
It is not a collection of bash, Drush, insert scripts here, etc.
+
DevOps is not about the tools... they don't fix everything magically But DevOps relies on some tools to implement its principles
DevOps: Why Is It Important
One-off environments and complicated manual builds and deployments = FAILURE
Must automate to meet business demands -- support agile, lean, faster, better...
Every environment is Production to someone
Drupal is not different -- all of these apply
+
A Classic Scenario
Jay: Will, you said this was working.
Will: It works in Dev.
Jay: Greg, can you look into why it isn't working in Prod?
Greg: Ok. (later on) @#$%! Dev and Prod are not the same!
+
Another Classic One
The Blog Post: Installing Drupal on AWS
1. Start wizard
2. 50+ steps later
3. A Drupal install
+
Why We Chose Chef
The orange color matches ours of course!
We already had experience (Marius) doing DevOps work with Chef in Bay Area with leading tech companies
We build Drupal applications -- we found Chef to be a good fit for application configuration management
+
The Operations part... ...out with the old and in with the new
+
Our Operations Story
How we started with Chef at Promet
Inconsistency was killing our time
How realistic was automation and configuration management of all of our stuff
+
Wasting Time
You are not in the sudoers file. This incident will be reported.
$ which git $
Fatal error: Class 'PDO' not found in blahblahblah
All of these messages diverted your work
+
Manual Config Run Amok
• Dev environments != Prod environments • Shared Dev/Prod/?? environments were
continually bloated • Over time modifications amounted to more
inconsistency between environments • Documentation was lacking
o No records explaining changes o Constant paper trail "paving" o No great place for it anyways (what wiki?!?)
• More customizations -> High Fragility
+
Typing Faster != Solution
• Runbooks, Bash and SSH o Process to stand up new infrastructure was done
manually via a "runbook”
• Any configuration time was repeated extensively
• Environment “discovery” bled time o Snooping around to analyze how things were initially
stood up and potentially modified (caching, etc.)
• Endless SSH-ing into servers
+
Chef Migration • In with the New
o 2 dedicated servers § VM hosts
o Similar model, consolidated o Entirely Chef managed o Individual client Chef-spun Dev instances
• Stand Up, QA, and Cutover o Virtualize legacy o Considerations mainly PHP
• Monitoring customizations • Special clients
o Non-Drupal o Contractually unique
+
Chef Migration
And migrating wasn't a complete nightmare....
+
Righting the Ship • Server configuration in Git
o Mitigated "paper trail" o Team awareness of configs by notifications o PHP, Apache, MySQL
• System config changes pushed to Chef for Server propagation o No more endless SSH-ing o No more runbook config for every server o Undoes "helpful" customizations by others
• Direct between me and Config Mgmt o Configuration entirely “local” with Vagrant o Git driven accountability and awareness (--stat) o `knife` is to Chef as drush is to Drupal
+
Client Ownership • Clients maintain ownership of their assets
o Any platform with a knife plugin o Legacy apps migrated o Non-Drupal or Promet "old" clients
• Give us their Keys o Let us be the Designated Driver o Developers drop in, get pitted
+
Client Ownership
• Excessive customization (not perfect) o Non-Drupal clients or other work o More so, use client's Hosted Chef
• Contract flexibility o No Dysfunctional Marriage
• Honor prior client / hosting relationships o Help clients where they are
+
Things That Are Solved • User Configs
o .gitconfig o .tmux.conf o .ssh/config o shell customizations o Not necessary but....doable
• Server Configs o logwatch o mail o mysql o apache o ssh o automysqlbackup
+
Sysadmin Transformation
Have time to explore/make new tools
Do support in a consistent fashion
Automate all the Things!
"Grow" your team and infrastructure not with numbers but with talent
We do more work with less resources!
+
Sysadmin Team Evolution
We even spun-off a Startup company for infrastructure and applications automation and scaling using Chef in the Bay Area (non-Drupal)
I don't care what you automate with... pick one and do it!
+
The Dev part... ...it's not what you think
+
What I Thought I Would Say
• What it was like... • How it sucked for Devs • How we changed it • How it sucked in new ways • How that lead to the new thing • Repeat.... • Where we are now • What's really fun about this for Devs
+
The Problem
• We went through too much shit • It's not even fun to talk about • It's probably boring • Someone else already said what I was trying
to justify • You're probably the choir if you're here
+
Drupal: OH @crell
"If you're not using Features, you are not doing Drupal professionally"
Also: • If you don't have a one step build, you aren't
using Features correctly => Sucking at doing Drupal professionally
• Configuration module is also probably fine • It's going into core
+
So Now My Presentation Is
• What we're doing now • Why you shouldn't NOT do what we're doing • Actual examples of how it has helped • Why Chef is really awesome • A surprise
+
What We're Doing Now
• Everything you need in the repo o Any scripts or scripting examples o README files o (small(-ish)) csv's/xml/etc o ...pretty much any text file that is not too large and is
useful
• ...Still need a way to automate sourcing archives of files and sql...
• Vagrantfile + Berksfile
+
Berks What?
+
+
Kirkberks? +
What a Dev Has to Do on Our Team
• Install git somehow • Install the vagrant package • Install the vagrant-berkshelf plugin • git clone/pull && cd-to-the-root • vagrant up (or reload) • edit the hosts file or use dnsmasq or similar
+
Some Concrete Examples of This Helping Us Do Better
• Audits o http://reload.github.io/phing-drupal-template o Not perfect... o But now no one has to repeat what is automated
• Every single deployment ever since o Your Production push should not be the first time
you attempt sync'ing to a Production environment o ...unless you crave excitement
• Last minute accommodations o Ever test RewriteBase?
+
Add the .htaccess directive... +
...change the docroot in code!
Potential Examples: You Can't Prove the Negative
• PECL/Pear packages o Shame on professionals who can't figure it out... o ...but shame on me for betting the project
• Anything to do with email ever since ever • Anything to do with external services
o ...memcached o ...SOLR o ...even MySQL
+
Why You Shouldn't NOT Do As We Do
Challenges you can deal with, excuses that are bullshit
On stupid things people say re: vagrant and chef
• We make great docs! • My native environment is fine • Virtualization consumes resources • I don't have enough time
+
+
My Native Environment is Fine
1. No, it isn't • Your job is to ship to your downstream
o ...not manicure and fetishize your tools
• You are getting strictly less help than you could get o ...even if you're getting a lot of help o ...even if you're paying for help
• You are keeping yourself from helping
+
My Native Environment is Fine (cont'd)
2. Your native environment is personal • You may even obsess over it • You may be completely wrong • You will be happier without each other
+
Virtualization Consumes Resources +
I Don't Have Time...
...to do everything perfectly
• We didn't either, and we're doing far better than we were
• We do not have a massive continuous delivery, cloud orchestration and scaling infrastructure
• ... But now what we have is in code
+
Chef is Really Awesome
...Puppet is probably cool too
• Stewardship and leadership • Constantly evolving ecosystem • A community hell-bent on sharing • Kicks ass even if you don't
+
So Easy I Can Share It with You!
Don't try it on this connection
bash <(curl -L -s http://bit.ly/10McZSZ)
github.com/promet/drupal_cookbook
Get to work already!
+
Why You Should Use this Project (or something like it)
We must advance the state of the art for Drupal deployment and application management
The CMS that is easiest to flexibly deploy will be the CMS of the future
All the features and D8 config exports in the world can't help you if you can't automate it
+
Other Projects/Strategies
• If you have a dope Jenkins workflow... o help me automate it with Chef so we can all use it!
• Please use Puppet if it suits you o and be vocal about hardcoded conventions that
make your work hard
• Kraftwagen - kraftwagen.org • Ariadne by patcon - github.com/
myplanetdigital/vagrant-ariadne
+
The Wrap-Up part... ...what should you remember
+
Key Summary Points
Process and team is the focus • Get your Devs and Sysadmins in the same
room and start communicating
Tools are there to help • Start automating now
Chef is an awesome tool • It works and it helps
+
Q&A
Ask us questions now
Talk to us afterward
Stop by and see us at Booth 201
Thanks!
+
Feedback? Evaluate session at: portland2013.drupal.org/schedule
top related