Functional Database Strategies

Functional Database Strategies

Scalæ by the Bay 2016

YesQL

by Jason Swartz@swartzrock


Step One

Buy A Functional Database

Silly! Databases Aremutable, not

functional!

Well?Aren’t they?

1. Immutable Rows2. Window Functions3. Events, Not State4. DB Interactions

Agenda


Agenda

Let’s Talk AboutImmutable Tables

Create-Read-Update-Delete

GET /issuesGET /issues/{id}POST /issuesPUT /issues/{id}

Issue Endpoints

How Do These EventsAffect The Database?

1. POST /issues title=’Config ELB’

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

1. POST /issues title=’Config ELB’2. PUT /issues/1 assignee=10

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+

1. POST /issues title=’Config ELB’2. PUT /issues/1 assignee=103. PUT /issues/1 done=true

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+



Not Bad.



Do You Know How We Got Here?








Why is ‘assignee’ NULL?

Mutable Table Rows Lose History

Immutable Table Rows Keep Their History

Let’s Try To Lock Down

Our State

1. POST /issues title=’Config ELB’

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

1. POST /issues title=’Config ELB’2. PUT /issues/1 assignee=10

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+


+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+


+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

1. GET /issues/1

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

Tables Are Mutable, But Table Rows Should Be

Immutable

In Other Words, Tables Should Be

Append-Only

How Do You Make An Append-Only

Table?

One: Don’t Let Your DB User Make

Changes

Grant select, insert on issues to my-db-user;

-- tested on Postgresql

Thank You! Goodbye!

Two: Pick The RightColumns

1. GET /issues/1

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

create table issues ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), title text, assignee int, done boolean default false)

1. GET /issues/1

+------+-------------+------------+------------+----------+-------+| id | created | issue_id | title | assignee | done |+------+-------------+------------+------------+----------+-------+| 1 | 09-18 18:13 | 1 | Config ELB | NULL | false |+------+-------------+------------+------------+----------+-------+| 2 | 09-18 18:16 | 1 | Config ELB | 10 | false |+------+-------------+------------+------------+----------+-------+| 3 | 09-18 18:19 | 1 | Config ELB | NULL | false |+------+-------------+------------+------------+----------+-------+

+------+-------------+------------+------------+----------+-------+| 4 | 09-18 18:24 | 1 | Config ELB | NULL | true |+------+-------------+------------+------------+----------+-------+

select * from issueswhere issue_id = :issue_idorder by id desc limit 1;

That’s The Basics Of Immutability

In Table Rows


Agenda


Agenda

SQL:2003 expands Groups into Windows

Works Great InPostgresql

● Aggregation functions, eg sum(), rank(), avg()● Window definitions with over()● Grouping with partition by, order

Window Functions

+------+------------+------------+----------+-------+| id | issue_id | title | assignee | done |+------+------------+------------+----------+-------+| 1 | 1 | Config ELB | NULL | false |+------+------------+------------+----------+-------+| 2 | 1 | Config ELB | 10 | false |+------+------------+------------+----------+-------+| 3 | 2 | Bug to fix | 11 | false |+------+------------+------------+----------+-------+| 4 | 2 | Bug to fix | 11 | true |+------+------------+------------+----------+-------+

+------+------------+------------+----------+-------+| id | issue_id | title | assignee | done |+------+------------+------------+----------+-------+| 1 | 1 | Config ELB | NULL | false |+------+------------+------------+----------+-------+| 2 | 1 | Config ELB | 10 | false |+------+------------+------------+----------+-------+| 3 | 2 | Bug to fix | 11 | false |+------+------------+------------+----------+-------+| 4 | 2 | Bug to fix | 11 | true |+------+------------+------------+----------+-------+

with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day')select * from latest_issues where row_number = 1




+------+------------+------------+----------+-------+------------+| id | issue_id | title | assignee | done | row_number |+------+------------+------------+----------+-------+------------+| 1 | 1 | Config ELB | NULL | false | 2 |+------+------------+------------+----------+-------+------------+| 2 | 1 | Config ELB | 10 | false | 1 |+------+------------+------------+----------+-------+------------+| 3 | 2 | Bug to fix | 11 | false | 2 |+------+------------+------------+----------+-------+------------+| 4 | 2 | Bug to fix | 11 | true | 1 |+------+------------+------------+----------+-------+------------+



That WasWindow

Functions


Agenda


Agenda

You Know How To Maintain State

Do We Still Need State?

Let’s Talk AboutEvent-Sourcing


+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | 10 | true |+------+-------------+------------+----------+-------+


+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | 10 | true |+------+-------------+------------+----------+-------+

Events

States

Now We’re Storing Events,

Not States

create table issue_events ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), originator text, payload text)

1. POST /issue/1/event ‘Originator: 4a48239-8a..’ payload=’<Update val=”done=true”>’

+----+-------------+----------+------------+---------+| id | created | issue_id | originator | payload |+----+-------------+----------+------------+---------+| 14 | 09-18 18:50 | 1 | 4a482... | <...> |+----+-------------+----------+------------+---------+

Create Events AndSimulate The State

1. Create-Issue

Issue(“Config ELB”, null, false);

Real Events

Virtual States

1. Create-Issue2. Assign-Issue

Issue(“Config ELB”, 10, false);

Real Events

Virtual States

1. Create-Issue2. Assign-Issue3. Complete-Issue

Issue(“Config ELB”, 10, true);

Real Events

Virtual States

So Why UseEvent-Sourcing?

1. High Write Performance2. Potential for Command/Query Separation3. Auditable4. Replayable5. Undo-able6. Monitorable

Reasons For Event-Sourcing

It’s Like Having Control Over The Versions Of

Your State Changes

It’s Like Having Control Over The Versions Of

Your Data

It’s Like GitFor Your Data

1. Frankly, It’s Weird2. Requires Events. No Events, No Event-Sourcing.3. As Of November 2016, It’s Still Non-Standard

Reasons Against Event-Sourcing

Wait! We’reScala

developers!

Who Cares About BeingNon-Standard?

That About Sums Up Event Sourcing


Agenda


Agenda

Remember That YourDatabase

is mutable.

Avoid SharingYour State

● Avoid shared mutable SESSIONS ● Avoid shared mutable CURSORS ● Mutating state? Cool! But MODEL IT FIRST● Execute state changes at THE EDGE

Safe Database Interactions

DoobieA Typelevel Project

Are these steps? Or a Monad?

That About Sums Up Database

Interactions

Okay, Actually That’s The Entire Talk

Unless There’s More Time

#


Scalæ by the Bay 2016

by Jason Swartz@swartzrock

Thank You For Attending

Fin

THIS SPACELEFT BLANK

Functional Database Strategies

Engineering