Functional Database Strategies at Scala Bay

Functional Database Strategies

Scala Bay • March 20, 2017

HelloScala Bay

HelloScala Bay

YesQL

by Jason Swartz@swartzrock


Step One

Buy A Functional Database

Silly! Databases Aremutable, not

functional!

Well?Aren’t they?

1. Functional DB’s In Scalaa. Doobie

2. Functional DB’s In The Databasea. Immutable Rowsb. Window Functionsc. Events, Not State

Agenda



Agenda



Agenda

Remember That YourDatabase

is mutable.

Avoid SharingYour State

● Avoid shared mutable SESSIONS ● Avoid shared mutable CURSORS ● Mutating state? Cool! But MODEL IT FIRST● Execute state changes at THE EDGE

Safe Database Interactions

DoobieA Typelevel Project

Are these steps? Or a Monad?

val xa: Transactor[Task] = DriverManagerTransactor[Task]( "org.postgresql.Driver", "jdbc:postgresql://localhost/issues", "u", "p")

case class Issue(id: Int, issueId: Int, title: String, assignee: Int, status: String)

val issue: Issue = sql""" select id, issue_id, title, assignee, status from issues where issue_id = ${req.issue_id} order by id desc limit 1 """ .query[Issue] .unique .transact(xa) .unsafePerformSync

● ResultSet to Case Class Conversion● Async IO modeling with Task (scalaz version)● Think in SQL, not an ORM● Plain SQL with sql"" interpolation● SQL functions supported

Why Doobie?



Agenda



Agenda

Let’s Talk AboutImmutable Tables

Create-Read-Update-Delete

GET /issuesGET /issues/{id}POST /issuesPUT /issues/{id}

Issue Endpoints

How Do These EventsAffect The Database?

1. POST /issues title=’Config ELB’

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

1. POST /issues title=’Config ELB’2. PUT /issues/1 assignee=10

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+

1. POST /issues title=’Config ELB’2. PUT /issues/1 assignee=103. PUT /issues/1 done=true

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+



Not Bad.



Do You Know How We Got Here?








Why is ‘assignee’ NULL?

Mutable Table Rows Lose History

Immutable Table Rows Keep Their History

Let’s Try To Lock Down

Our State

1. POST /issues title=’Config ELB’

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

1. POST /issues title=’Config ELB’2. PUT /issues/1 assignee=10

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+


+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+


+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

1. GET /issues/1

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

Tables Are Mutable, But Table Rows Should Be

Immutable

In Other Words, Tables Should Be

Append-Only

How Do You Make An Append-Only

Table?

One: Don’t Let Your DB User Make

Changes

Grant select, insert on issues to my-db-user;

-- tested on Postgresql

Thank You! Goodbye!

Two: Pick The RightColumns

1. GET /issues/1

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

create table issues ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), title text, assignee int, done boolean default false)

1. GET /issues/1

+------+-------------+------------+------------+----------+-------+| id | created | issue_id | title | assignee | done |+------+-------------+------------+------------+----------+-------+| 1 | 09-18 18:13 | 1 | Config ELB | NULL | false |+------+-------------+------------+------------+----------+-------+| 2 | 09-18 18:16 | 1 | Config ELB | 10 | false |+------+-------------+------------+------------+----------+-------+| 3 | 09-18 18:19 | 1 | Config ELB | NULL | false |+------+-------------+------------+------------+----------+-------+

+------+-------------+------------+------------+----------+-------+| 4 | 09-18 18:24 | 1 | Config ELB | NULL | true |+------+-------------+------------+------------+----------+-------+

select * from issueswhere issue_id = :issue_idorder by id desc limit 1;

What Defines AnIssue,

The Version or the Entity?

ISSUES+------+-------------+------------+------------+----------+-------+| id | created | issue_id | title | assignee | done |+------+-------------+------------+------------+----------+-------+| 1 | 09-18 18:13 | 1 | Config ELB | NULL | false |+------+-------------+------------+------------+----------+-------+| 2 | 09-18 18:16 | 1 | Config ELB | 10 | false |+------+-------------+------------+------------+----------+-------+| 3 | 09-18 18:19 | 1 | Config ELB | NULL | false |+------+-------------+------------+------------+----------+-------+

create table issues ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), title text, assignee int, done boolean default false )

create table notifications ( id serial, created timestamp default now(), issue_row_id int references issues )


NOTIFICATIONS+------+-------------+--------------+| id | created | issue_row_id | +------+-------------+--------------+| 1 | 09-18 19:13 | 2 |+------+-------------+--------------+


NOTIFICATIONS+------+-------------+--------------+| id | created | issue_row_id | +------+-------------+--------------+| 1 | 09-18 19:13 | 2 |+------+-------------+--------------+

Issue Notifications AreImmutableIn The Database

That’s The Basics Of Immutability

In Table Rows



Agenda



Agenda

SQL:2003 expands Groups into Windows

Works Great InPostgresql

● Aggregation functions, eg sum(), rank(), avg()● Window definitions with over()● Grouping with partition by, order

Window Functions

+------+------------+------------+----------+-------+| id | issue_id | title | assignee | done |+------+------------+------------+----------+-------+| 1 | 1 | Config ELB | NULL | false |+------+------------+------------+----------+-------+| 2 | 1 | Config ELB | 10 | false |+------+------------+------------+----------+-------+| 3 | 2 | Bug to fix | 11 | false |+------+------------+------------+----------+-------+| 4 | 2 | Bug to fix | 11 | true |+------+------------+------------+----------+-------+

+------+------------+------------+----------+-------+| id | issue_id | title | assignee | done |+------+------------+------------+----------+-------+| 1 | 1 | Config ELB | NULL | false |+------+------------+------------+----------+-------+| 2 | 1 | Config ELB | 10 | false |+------+------------+------------+----------+-------+| 3 | 2 | Bug to fix | 11 | false |+------+------------+------------+----------+-------+| 4 | 2 | Bug to fix | 11 | true |+------+------------+------------+----------+-------+

with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day')select * from latest_issues where row_number = 1




+------+------------+------------+----------+-------+------------+| id | issue_id | title | assignee | done | row_number |+------+------------+------------+----------+-------+------------+| 1 | 1 | Config ELB | NULL | false | 2 |+------+------------+------------+----------+-------+------------+| 2 | 1 | Config ELB | 10 | false | 1 |+------+------------+------------+----------+-------+------------+| 3 | 2 | Bug to fix | 11 | false | 2 |+------+------------+------------+----------+-------+------------+| 4 | 2 | Bug to fix | 11 | true | 1 |+------+------------+------------+----------+-------+------------+



That WasWindow

Functions



Agenda



Agenda

You Know How To Maintain State

Do We Still Need State?

Let’s Talk AboutEvent-Sourcing


+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | 10 | true |+------+-------------+------------+----------+-------+


+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | 10 | true |+------+-------------+------------+----------+-------+

Events

States

Now We’re Storing Events,

Not States

create table issue_events ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), originator text, payload text)

1. POST /issue/1/event ‘Originator: 4a48239-8a..’ payload=’<Update val=”done=true”>’

+----+-------------+----------+------------+---------+| id | created | issue_id | originator | payload |+----+-------------+----------+------------+---------+| 14 | 09-18 18:50 | 1 | 4a482... | <...> |+----+-------------+----------+------------+---------+

Create Events AndSimulate The State

1. Create-Issue

Issue(“Config ELB”, null, false);

Real Events

Virtual States

1. Create-Issue2. Assign-Issue

Issue(“Config ELB”, 10, false);

Real Events

Virtual States

1. Create-Issue2. Assign-Issue3. Complete-Issue

Issue(“Config ELB”, 10, true);

Real Events

Virtual States

So Why UseEvent-Sourcing?

1. High Write Performance2. Potential for Command/Query Separation3. Auditable4. Replayable5. Undo-able6. Monitorable

Reasons For Event-Sourcing

It’s Like Having Control Over The Versions Of

Your State Changes

It’s Like Having Control Over The Versions Of

Your Data

It’s Like GitFor Your Data

1. Frankly, It’s Weird2. Requires Events. No Events, No Event-Sourcing.3. As Of March 2017, It’s Still Non-Standard

Reasons Against Event-Sourcing

Wait! We’reScala

developers!

Who Cares About BeingNon-Standard?

That About Sums Up Event Sourcing

That About Sums Up Database

Interactions

Okay, Actually That’s The Entire Talk

Unless There’s More Time

#


Scala Bay • March 20, 2017

by Jason Swartz@swartzrock

Thank You For Attending

Fin

THIS SPACELEFT BLANK

Functional Database Strategies at Scala Bay

Technology