javazone 2014

Database migrations - the missing piece in Java EE

Rikard Thulin

3 billion Java developers lives was ruined on

August 29, 1997

The survivors of the deployment called the war

Judgment Day.

They lived only to face a new nightmare, the war against the

DBA.

Skynet, the computer which controlled releases, sent

two engineers back through time

Their mission: to solve the problems of

schema migrations…

Opening “quote” from “Terminator 2: Judgment Day”

You will learn how to

be happier as you can spend

more time writing code!

And we will have a look into

the future and the

possibilities at hand

Roadmap

• Introduction to the topic and why it is so important

• The current situation, the tools of the trade

• Future possibilities at hand to solve this problem

Part 1

Introduction to the topic and why it is so important

Finally, we where ready to go

IT WAS RELEASE DAY

Just get the luggage

There might be a small burden

Let the team lead or the smart guy do it!Or if you are a manager, it translates to the expensive guy

The team goes…

This is your complete team

change recall refactoring applied some marked specified better run units the tagged along more SQL precondition pain state both statements contains production Sets applying database nested things executes identified context leave test one management Generally apply HOPE change stable individual ran database just IT generate fails different changes uniquely per machines change written productivity log attribute Set change sets development author files WORKS Contexts describes by descript ive someone regardless Major else environments column queries local Each yet either WORK change ANYMORE set Preconditions DATABASECHANGELOG unexpected control file !? Concepts stop store execution arbitrarily location auto commit executed Developers



Finally we are set to go…

…but…

DBA

Fill in the forms…

You are trapped in the Jar(aka jar-hell)

It breaks andYou look like this

You gently communicate with your fellow co-worker

This was embarrassing and unprofessional

Time to be a hero!

Engineers likes to be heroes

This has to be a good thing, right?

You (the smart guy) spend the weekend

Your boss turns to thisAre you nuts? OVERTIME?!

The obvious solution to professionally solve the problem

We add pre-release meeting!

change remember refactoring applied some marked specified better run units the tagged along more SQL precondition and state both statements contains production Sets last database nested things executes identified context leave test one management Generally apply suddenly change stable individual ran time just a generate fails different STUFF uniquely per machines change co-worker productivity log attribute Set change sets development author files DOES Contexts describes added descriptive NOT regardless Major a environments column queries local Each yet either WORK change ANYMORE set Preconditions DATABASECHANGELOG unexpected control file !? Concepts stop store execution arbitrarily location auto commit executed Developers



$ git pull



Instead

The aha moment

From

To

The smiling team

Productivity increased

You will learn how tobe happier as you can spend

more time writing code!

Sweden

Friendly

neighbor

to

Norway

unlike

Norway

we

do

not

have

a

storage

problem

Proven national gas reserves (cubic meters)

0 2,007,000,000,000

Proven national oil reserves (barrels) 0 5,320,000,000

NO STORAGE PROBLEM

Development

One developer, life is easy!

Development

Obvious we need production

Production

Anotherjoins

Now each developer can work

Development Production

Time to bring out the engineer

from the closet

Development ProdCI

for the engineers to cooperate

we need Continuos Integration

Time to really

bring out the engineerfrom the closet

ProdDev CI

someone told the boss that things should not only be tested

on end users

QA

Time to really, really

bring out the engineerfrom the closet

StagingDev CI

Now when we know about quality assurance we should stage

our production environment

QA

Prod

Morejoins

As we do everything by the book

things ($) goes well

Staging clusterCI

How do we keep everything CONSISTENT?

QA

Staging cluster

Prod clusterProd cluster

Dev

{code}VCS Build server

How do we keep { code }

CONSISTENT?

artifactsRepository

push pull

publish

release

Dev

QA

promote

DevStagingProd

How do we keep { code + database }

CONSISTENT?

Two entities

Really?Is the database EVIL?

artifact manual, script, …

Source Database

the problem is to keep

them consistent

HOW

artifact manual, script, …

Source Database

db deltas, automatic

artifact

Source

{code +

migrations}

VCS Build server

How do we keep { code + database }

CONSISTENT?

artifacts

{code + migrations}

Repository

push pull

publish

release

Dev

QA

promote

DevStagingProd

Shiny Happy PeopleYou will get

Schema MigrationDEMO DEMO DEMO DEMO


Migrations are applied automatically during deployment


Spring based application


What you will see

Shiny Happy PeopleDEMONSTRATION OF

LOCAL DEV PROD

Version 0 Version 0 Version 0

LOCAL DEV PROD


LOCAL DEV PROD


LOCAL DEV PROD


LOCAL DEV PROD


LOCAL DEV PROD


What can we do?create/drop TABLEcreate/drop INDEX

add/remove CONSTRAINT alter SQUENCE

STOREDPROCEDUREScreate/drop VIEWS

insert DATA

and more!

What

is the trick?

When

is the trick?

Database Change ManagementReal Life Tricks

• Possible to use on an existing database • Quick and simple to implement • Database copies mix and match • Wash copies of production database • DBA approval / review process • One click release possible

Part 2

The current situation tools of the trade

Tools of the Trade

ActiveRecord Ruby (Ruby)

Entity Framework Migrations (.net)

COMPETITION HAS IT

Tools of the Trade

Hibernate & JPA Schema Generation

Tools of the Trade

FlyWay & LiquibaseStronger execution order Supports different DB / environment Stereotype of being “heavyweight" Automatic rollback

Convention over configuration Gaining popularity

Tools of the TradeFlyWay

Convention over configuration Get up and running in 5 minutes Gaining popularity

Tools of the TradeLiquibase

Stronger execution order Supports different DB for different environment Stereotype of being “heavyweight" Automatic rollback

Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one

of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the

FlyWay & Liquibase Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one

of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one of the tools Use one

Tools of the Trade summary

Schema migration is •Not solved by Java EE today •Solved by 3PP •Already in competitors standards

Part 3

Future possibilities to solve this problem

Reasons to add Migrations to Java EE

We are behind the competition Schema migrations is not well know to the broad developer community Increased Quality and Productivity

Adding complexity to the platform

Advanced tooling

the future?

class User { }

@Migration(id="1", author="duke") class CreateUserTable {

@CreateTable(User.class) @AddColumn(User.class) long id;

@AddColumn(entity = User.class, size = "255") String name; }

the future?

the future?

@Migration(id="1", author="duke") class CreateUserTable { @AddColumn(User.class) long id; … }

@Entity class User { @Id long id; … }

Migration2Code Generator

@Entity class User {

@Id long id;

@Column(size="255") String name; }

the future?

@Migration(id="2", author="duke") class AddColumnsToUser {

@DropColumn(User.class) String name;

@AddColumn(entity = User.class, size="80") String firstName;

@AddColumn(entity = User.class, size="80") String lastName;

@AddColumn(entity = User.class, size="255") String email; }

the future?

@Entity class User {

@Id long id;

@Column(size="80") String firstName;

@Column(size="80") String lastName;

@Column(size="255") String email; }

the future?

The future my proposal

Add schema migration to

Java EE

The future my proposal

Who? WE JCP JSR! What? Lets figure it out!When? Now. Lets start!

Wrap up

Wrap up

1. Start using one of the tools! 2. We need to figure out if

database migrations is a candidate to be included in Java EE

3. Now it the time to take action!

You can be the Hero

Hero wrap up

Q & A

[email protected]

javazone 2014

Documents