Getting CI right for SQL Server

@_AlexYates_#ManSSUG

Getting CI right for SQL Server

Alex Yates

@_AlexYates_#ManSSUG

DLM Consultant

workingwithdevs.com

[email protected]

Alex Yates

@_AlexYates_

@_AlexYates_#ManSSUG

@_AlexYates_ | #ManSSUG

@_AlexYates_#ManSSUG

Farm Credit Services of America (FCSA)

• 100 person IT team, 14 sub-teams

• Database version control inconsistent

• Deployment process manual

• Delivery was slow and unreliable

@_AlexYates_#ManSSUG

What is CI?

@_AlexYates_#ManSSUG

What is CI?

Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.

www.thoughtworks.com/continuous-integration

@_AlexYates_#ManSSUG

http://www.jamesshore.com/Blog/Continuous-Integration-on-a-Dollar-a-Day.html

@_AlexYates_#ManSSUG

Farm Credit Services of America (FCSA)

• Growing test suite

• Delivery faster and more reliable

• Standardised source control process

• Automated builds/deploys from source control

@_AlexYates_#ManSSUG

The deployment pipeline

@_AlexYates_#ManSSUG

Databases are hard

• Schema changes vs existing data• Reference data vs production data• Teamwork and testing• Database drift (change outside

process, e.g. production hot-fixes)

@_AlexYates_#ManSSUG

The deployment pipeline

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

Version Control

@_AlexYates_#ManSSUG

Version control - LMGTFY

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

There’s more than one way to skin a cat

@_AlexYates_#ManSSUG

There’s more than one way to skin automatea cat

@_AlexYates_#ManSSUG

V1 V2

@_AlexYates_#ManSSUG

V1 V2

Migrations-based solutions

@_AlexYates_#ManSSUG

V1 V2

State-based solutions

@_AlexYates_#ManSSUG

@_AlexYates_#ManSSUG

“There's nothing more reliable than keeping track of exactly the scripts you intend to run, and running them, without trying to compare state and guess.”

@_AlexYates_#ManSSUG

“There's nothing more reliable than keeping track of exactly the scripts you intend to run, and running them, without trying to compare state and guess.”

Paul Stovell, built Octopus Deploy

http://docs.octopusdeploy.com/display/OD/SQL+Server+databases

@_AlexYates_#ManSSUG

“As soon as you have multiple changes on a single aspect of an object, ordering and the ability to detect which change needs to be made gets very complicated.”

@_AlexYates_#ManSSUG

“As soon as you have multiple changes on a single aspect of an object, ordering and the ability to detect which change needs to be made gets very complicated.”

Gert Drapers, built DataDude

https://blogs.msdn.microsoft.com/gertd/2009/06/05/declarative-database-development/

@_AlexYates_#ManSSUG

Migrations vs state

http://workingwithdevs.com/delivering-databases-migrations-vs-state/

@_AlexYates_#ManSSUG

Migrations vs state

http://workingwithdevs.com/delivering-databases-migrations-vs-state/

@_AlexYates_#ManSSUG

Migrations vs state

http://workingwithdevs.com/delivering-databases-migrations-vs-state/

@_AlexYates_#ManSSUG

Migrations vs state

http://workingwithdevs.com/delivering-databases-migrations-vs-state/

@_AlexYates_#ManSSUG

Migrations vs state

http://workingwithdevs.com/delivering-databases-migrations-vs-state/

@_AlexYates_#ManSSUG

Migrations vs state

http://workingwithdevs.com/delivering-databases-migrations-vs-state/

@_AlexYates_#ManSSUG

Migrations vs state

http://workingwithdevs.com/delivering-databases-migrations-vs-state/

@_AlexYates_#ManSSUG

State Easier (less control) Better for

sprocs/functions Better for

large/distributed teams Better for frequent

changes Better for dependency

nightmares Drift: rolled back Better for development

Migrations More control

(harder/needs discipline) Better for data

migrations Better for small teams Better for infrequent

changes Better for simple data

stores Drift: ignored Better for automation

VS

@_AlexYates_#ManSSUG

DLMConsultants.com/model-vs-mig

@_AlexYates_#ManSSUG

Demo

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

Continuous Integration

@_AlexYates_#ManSSUG

Build servers

@_AlexYates_#ManSSUG

VCS Build Test Sync

Upgrade scriptDatabase package

Test results

The automated build process

@_AlexYates_#ManSSUG

Builds/Tests are run on CI agents

Builds/Tests are managed on

CI server

CI server

How do build servers work?

@_AlexYates_#ManSSUG

Demo

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

Unit testing

@_AlexYates_#ManSSUG

Unit testing

www.tsqlt.org

@_AlexYates_#ManSSUG

Unit testing

www.tsqlt.org

@_AlexYates_#ManSSUG

Unit testing

www.tsqlt.org

@_AlexYates_#ManSSUG

www.tsqlt.org

Unit testing

@_AlexYates_#ManSSUG

Unit testing

@_AlexYates_#ManSSUG

Unit testing

@_AlexYates_#ManSSUG

http://workingwithdevs.com/three-odd-attitudes-towards-database-unit-testing-that-make-me-angry/

@_AlexYates_#ManSSUG

Demo

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

Drift

@_AlexYates_#ManSSUG

@_AlexYates_#ManSSUG

“When making changes directly on production, you are making a decision that the delay due to poor cycle time is more expensive than the risk of making a mistake.”

@_AlexYates_#ManSSUG

“When making changes directly on production, you are making a decision that the delay due to poor cycle time is more expensive than the risk of making a mistake.”

Alex Yates, built this slide

@_AlexYates_#ManSSUG

The bad stuff:

• Accidental roll-backs (state)

• Failed deployments (migrations)

• Environment inconsistency (migrations)

@_AlexYates_#ManSSUG

DriftStrategies to help

Improve cycle time Strict security policies Monitor drift (play with DDL

triggers) Redgate DLM Dashboard

(free)https://www.simple-talk.com/sql/database-administration/database-deployment-the-bits-database-version-drift/

@_AlexYates_#ManSSUG

DriftStrategies to help

Improve cycle time Strict security policies Monitor drift (play with DDL

triggers) Redgate DLM Dashboard

(free)http://www.red-gate.com/products/dlm/dlm-dashboard/

@_AlexYates_#ManSSUG

Demo

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

• Teamwork above tools/process• Don’t do *all the things* at once - start with VCS and drift• Don’t forget tests• Decouple database from other systems• Learn CI on an easy database/project• Use the right process/tool for your database and team

• Ask me for help!

General advice

@_AlexYates_#ManSSUG

AgendaVersion control

Model vs migrationsAutomated builds

Unit testingDrift

General advice

@_AlexYates_#ManSSUG

• Standardised VCS, CI and RM processes• FCSA delivering much more efficiently

• FCSA process model for sister organisations

• The team all have excellent CV’s

Farm Credit Services of America (FCSA)

@_AlexYates_#ManSSUG

• Standardised VCS, CI and RM processes• FCSA delivering much more efficiently

• FCSA process model for sister organisations

• The team all have excellent CV’s

Farm Credit Services of America (FCSA)

http://www.codeaperture.io/2016/09/13/how-redgate-helped-define-our-process/

@_AlexYates_#ManSSUG

Siloed sparrows suck at DLM

@_AlexYates_#ManSSUG

Heated hippos are closed minded

@_AlexYates_#ManSSUG

State Easier (less control) Better for

sprocs/functions Better for

large/distributed teams Better for frequent

changes Better for dependency

nightmares Drift: rolled back Better for development

Migrations More control

(harder/needs discipline) Better for data

migrations Better for small teams Better for infrequent

changes Better for simple data

stores Drift: ignored Better for automation

VS

Clever people consider options

@_AlexYates_#ManSSUG

Use appropriate tools

@_AlexYates_#ManSSUG

Only hire developers who write tests

@_AlexYates_#ManSSUG

Hunt his smug face - make pain on it

@_AlexYates_#ManSSUG

This stuff makes a big difference

http://www.codeaperture.io/2016/09/13/how-redgate-helped-define-our-process/

@_AlexYates_#ManSSUG

DLMConsultants.com/ci

@_AlexYates_#ManSSUG

Questions?

Links: DLMConsultants.com/ciContact me: [email protected]

@_AlexYates_#ManSSUG

Image sourcesAuthor Source Information

Chiltepinster Wikimedia Commons Mocking Bird Argument.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license. Source on Wikimedia Commons: “Own work”

Bit Boy Flickr The elephant in the room – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.

Nils Rinaldi Flickr Hippo fight 2/3 – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.

My own collection Taken by/property of Alex Yates Kitten, “There’s more than one way to skin a cat!”

Memegenerator.net Memegenerator.net I don’t always edit database. Content designed to be shared and delivered with credit to memegenerator.net.

Ctrl.Alt.Design ctrla.lt Social Media share icons