Developing functional domain models with event sourcing (sbtb, sbtb2015)

@crichardson

Developing functional domain models with event sourcingChris Richardson

Author of POJOs in Action Founder of the original CloudFoundry.com Founder of a microservices platform startup

@crichardson [email protected] http://plainoldobjects.com http://microservices.io

@crichardson

Presentation goal

How to design functional domain models using event sourcing?

@crichardson

About Chris

@crichardson

About Chris

Consultant and trainer focusing on microservices (http://www.chrisrichardson.net/)

@crichardson

About Chris

Founder of a startup that is creating a platform that makes it easy for

application developers write microservices

(http://bit.ly/trialeventuate)

@crichardson

For more information

https://github.com/cer/event-sourcing-examples

http://microservices.io

http://plainoldobjects.com/

https://twitter.com/crichardson

@crichardson

Agenda

Why event sourcing?

Designing a domain model based on event sourcing

Event sourcing and service design

@crichardson

Tomcat

Traditional monolithic architecture

Browser/Client

WAR/EAR

RDBMS

Customers

Accounts

Transfers

Banking UI

develop test

deploy

Simple to

Load balancer

scale

Spring MVC

Spring Hibernate

...

HTML

REST/JSON

ACID

@crichardson

But that leads* to monolithic hell

For large and/or complex applications…

@crichardson

Today: use a microservice, polyglot architecture

Banking UI

Account Management Service MoneyTransfer Management Service

Account Database MoneyTransfer Database

Standalone services

Sharded SQLNoSQL DB

@crichardson

But now we have distributed data

management problems

@crichardson

Customer management

Maintaining invariants across service boundaries

Order management

Order Service

placeOrder()

Customer Service

updateCreditLimit()

Customer

creditLimit ...

has ordersbelongs toOrder

total

Invariant: sum(open order.total) <= customer.creditLimit

?

@crichardson

Maintaining invariants with (non-ACID) NoSQL databases

Non-ACID NoSQL database

Order Service

placeOrder()

Customer Service

updateCreditLimit()

Customer

creditLimit ...

has ordersbelongs toOrder

total

Invariant: sum(open order.total) <= customer.creditLimit

?

@crichardson

Use an event-driven architecture

Services publish events when state changes

Services subscribe to events and update their state

Maintain eventual consistency across multiple aggregates (in multiple datastores)

Synchronize replicated data

@crichardson

Order ManagementOrder

id : 4567 total: 343 state = CREATED

Customer Management

Customer creditLimit : 12000 creditReservations: {}

Customer creditLimit : 12000 creditReservations: { 4567 -> 343}

Order id : 4567 total: 343 state = OPEN

Eventually consistent credit checking

Message Bus

createOrder()

Publishes:Subscribes to:

Subscribes to:

publishes:

OrderCreatedEvent

CreditReservedEvent

OrderCreatedEvent CreditReservedEvent

@crichardson

How to atomically

update the database and

publish events without 2PC?

(dual write problem)

@crichardson

Update the database and

Publish events

@crichardson

Event sourcingFor each aggregate (aka. business entity):

Identify (state-changing) domain events

Define Event classes

For example,

Account: AccountOpenedEvent, AccountDebitedEvent, AccountCreditedEvent

ShoppingCart: ItemAddedEvent, ItemRemovedEvent, OrderPlacedEvent

@crichardson

Persists events NOT current state

Account

balance

open(initial) debit(amount) credit(amount)

AccountOpened

Event table

AccountCredited

AccountDebited

101 450

Account tableX101

101

101

901

902

903

500

250

300

@crichardson

Replay events to recreate state

Account

balance

AccountOpenedEvent(balance) AccountDebitedEvent(amount) AccountCreditedEvent(amount)

Events

Periodically snapshot to avoid loading all events

@crichardson

The present is a fold over history

@crichardson

Event

Aggregates: Command => Events

AggregateCommand Event

@crichardson

Request handling in an event-sourced application

HTTP Handler

Event Store

pastEvents = findEvents(entityId)

Account

new()

applyEvents(pastEvents)

newEvents = processCmd(SomeCmd)

saveEvents(newEvents)

Microservice A

(optimistic locking)

@crichardson

Event Store publishes events - consumed by other services

Event Store

Event Subscriber

subscribe(EventTypes)

publish(event)

publish(event)

Aggregate

CQRS View

update()

update()

Microservice B

send notifications

…

@crichardson

Benefits of event sourcingSolves data consistency issues in a Microservice/NoSQL-based architecture

Reliable event publishing: publishes events needed by predictive analytics etc, user notifications,…

Eliminates O/R mapping problem (mostly)

Reifies state changes:

Built-in, reliable audit log,

temporal queries

Preserved history ⇒ More easily implement future requirements

@crichardson

Drawbacks of event sourcing

Weird and unfamiliar

Events = a historical record of your bad design decisions

Handling duplicate events can be tricky

Application must handle eventually consistent data

Event store only directly supports PK-based lookup => use Command Query Responsibility Segregation (CQRS) to handle queries

@crichardson

Agenda

Why event sourcing?

Designing a domain model based on event sourcing

Event sourcing and service design

@crichardson

Use the familiar building blocks of DDD

Entity

Value object

Services

Repositories

Aggregates

@crichardson

Partition the domain model into Aggregates

Order

OrderLine Item

quantity

…

Addressstreet city …

Customer

Productname price

Aggregate designGraph consisting of a root entity and one or more other entities and value objects

Each core business entity = Aggregate: e.g. customer, Account, Order, Product, ….

Reference other aggregate roots via primary key

Often contains partial copy of other aggregates’ data

Order

OrderLine Item

quantity productId productName productPrice

customerId

Address

street city …

@crichardson

Modular business logic = flexible deployment

Tomcat

WAR/EAR

Customer

Order

Tomcat

WAR/EAR

Customer

Tomcat

WAR/EAR

Order

OR

@crichardson

Aggregate granularity is important

Transaction = processing one command by one aggregate

No opportunity to update multiple aggregates within a transaction

If an update must be atomic (i.e. no compensating transaction) then it must be handled by a single aggregate

e.g. scanning boarding pass at security checkpoint or when entering jetway

@crichardson

Aggregate granularity

Customer

Product

Order

Customer

Product

Order

ConsistencyDecomposability/

Scalability/ User experience

Designing domain eventsRecords state changes for an aggregate

Records key “business events”

Part of the public API of the domain model ProductAddedToCart

id : TimeUUID productId productName productPrice shoppingCartId

Required by aggregate

Enrichment: Required by consumers

@crichardson

Designing commandsCreated by a service from incoming request

Processed by an aggregate

Immutable

Contains value objects for

Validating request

Creating event

Auditing user activity

@crichardson

Hybrid OO/FP domain objects

@crichardson

OO = State + Behavior

creditLimit creditReservations

Customer

processCommand : PartialFunction[Command, Events]

applyEvent : PartialFunction[Event, Account]

State

Behavior

@crichardson

Aggregate traits

Map Command to Events

Apply event returning updated Aggregate

Used by Event Store

to reconstitute aggregate

Event types are not precisely typed :-(

@crichardson

Customer - command processing

Check available credit

@crichardson

Customer - applying events

Immutable

@crichardson

Event Store APItrait EventStore {

def save[T <: Aggregate[T]](entity: T, events: Seq[Event], assignedId : Option[EntityId] = None): Future[EntityWithIdAndVersion[T]]

def update[T <: Aggregate[T]](entityIdAndVersion : EntityIdAndVersion, entity: T, events: Seq[Event]): Future[EntityWithIdAndVersion[T]]

def find[T <: Aggregate[T] : ClassTag](entityId: EntityId) : Future[EntityWithIdAndVersion[T]]

def findOptional[T <: Aggregate[T] : ClassTag](entityId: EntityId) Future[Option[EntityWithIdAndVersion[T]]]

def subscribe(subscriptionId: SubscriptionId): Future[AcknowledgableEventStream] }

@crichardson

Unsatisfying design with imprecise typing

https://flic.kr/p/2Bh518

“I hate OO”

@crichardson

A detour to Haskell

http://www.meetup.com/oakland-scala/Oakland Advanced Scala Study Group

https://www.flickr.com/photos/georgikeith/2658949664

@crichardson

Haskell event sourcing aggregate…

https://gist.github.com/Fristi/7327904

Associated data family ~ abstract type members

The operations

@crichardson

…Haskell Aggregate

@crichardson

Haskell TicTacToe aggregateConcrete type definitions

Command Processing

Apply Event

@crichardson

Functional event sourcing in Scala

@crichardson

“This section briefly explains GHC Haskell’s associated types and shows in detail how they can be encoded in Scala using

type members …”

@crichardson

FP = Separation of State and Behavior

Customer

creditLimit …

CustomerAggregate

processCommand(Account, Command) : Seq[Events]

applyEvent(Account, Event) : Account

State Behavior

@crichardson

Aggregate type classesUsed by

Event Store to

reconstitute aggregates

Hardwired

@crichardson

Customer Aggregate….State

Behavior

@crichardson

…command processing…

@crichardson

… applying events

@crichardson

Using Lenses with aggregates

@crichardson

FP-style Event Store API

Aggregate type class

@crichardson

Agenda

Why event sourcing?

Designing a domain model based on event sourcing

Event sourcing and service design

@crichardsonEvent Store

HTTP Request

HTTP Adapter

Event Handler

Cmd

Cmd

Events

Events

Xyz Adapter

Xyz Requestmicroservice

Aggregate

Service

Event Adapter

@crichardson

Place Order example

As a customer I want to place an order So that I get the needed products

Given that my available credit is $1500 When I place a $250 order Then the order is created Then my available credit is $1250

Story

Scenario

Post conditions

Pre conditions

@crichardson

Old-style ACID…BEGIN TRANSACTION

… RESERVE CREDIT …

… CREATE ORDER…

COMMIT TRANSACTION

… becomes eventually consistent (BASE)

Updating multiple aggregates

multi-step, event-driven flow

each step updates one Aggregate

Service creates saga to coordinate workflow

A state machine

Part of the domain, e.g. Order aggregate OR Synthetic aggregate

Post-conditions eventually true

Order

Customer

CreatedCredit reserved

public Order createOrder() { … Creates Order … }

Approved

@crichardson

Need compensating transactions

Pre-conditions might be false when attempting to update an aggregate

Credit check might fail => cancel order

Credit check succeeded but customer cancels order => undo credit reservation

…

@crichardson

Creating an order

DSL concisely specifies: 1.Creates Customer aggregate 2.Processes command 3.Applies events 4.Persists events

@crichardson

Event handling in AccountDurable subscription nameTriggers BeanPostProcessor

1.Load Customer aggregate 2.Processes command 3.Applies events 4.Persists events

@crichardson

SummaryEvent sourcing solves a variety of problems in modern application architectures

ES-based architecture = choice of monolith or microservices

Scala is a great language for implementing ES-based domain models:

Case classes

Pattern matching

Recreating state = functional fold over events

@crichardson

@crichardson [email protected]

http://plainoldobjects.com http://microservices.io http://bit.ly/trialeventuate