CONFIDENTIAL Designator
1
MIcroservices Data Patterns
CQRS and Event Source with Kafka and Eclipse Vert.x
Don Schenckdigital evangelist
CONFIDENTIAL Designator
Code is easy...
2
Unless it’s RPG III
CONFIDENTIAL Designator
Code is easy...
...Data is HARD
3
CONFIDENTIAL Designator
Data Everywhere:
4
CONFIDENTIAL Designator
Data Everywhere:RDBMS
5
CONFIDENTIAL Designator
Data Everywhere:RDBMSNoSQL
6
CONFIDENTIAL Designator
Data Everywhere:RDBMSNoSQLFlat files
7
CONFIDENTIAL Designator
Data Everywhere:RDBMSNoSQLFlat files (including spreadsheets!)
8
Ugh. Make it stop.
CONFIDENTIAL Designator
Centralized App + Centralized Data = “Everything’s fine”
9
CONFIDENTIAL Designator
But then ...
10
CONFIDENTIAL Designator
11
CONFIDENTIAL Designator
12
CONFIDENTIAL Designator
13
CONFIDENTIAL Designator
14
Remember: The “S” in IoT stands for “Security”
CONFIDENTIAL Designator
https://developers.redhat.com/books/migrating-microservice-databases-relational-monolith-distributed-data/
CONFIDENTIAL Designator
Split data from RDBMS
CONFIDENTIAL Designator
Apps change; data tends to stick around
CONFIDENTIAL Designator
You always have data.
CONFIDENTIAL Designator
A decade (or so) ago:ORM - HibernatePOJOs and XML
CONFIDENTIAL Designator
Then … Event Sourcing came along
CONFIDENTIAL Designator
Think “Events” instead of “Data Structure”.
CONFIDENTIAL Designator
ID customerID balance
1001 990 1000
1002 991 0
1003 991 -500
1004 992 300
Account
CONFIDENTIAL Designator
{
"event": "transferFunds",
"fromAccountID": 1001,
"toAccountID": 1001,
"amount": 500.00,
"timestamp": "2019-08-14T11:05:27:23.000212"
}
CONFIDENTIAL Designator
An Event is based on the real world, not IT-think.
CONFIDENTIAL Designator
CQSCommand-Query Separation
CONFIDENTIAL Designator
“Asking a question should not change the answer” -- Bertrand Meyer
CONFIDENTIAL Designator
CQRS - Command Query Responsibility Segregation
CONFIDENTIAL Designator
A simple, basic example
CONFIDENTIAL Designator
This is simple?
CONFIDENTIAL Designator
But distributed systems bring complexity...
CONFIDENTIAL Designator
CONFIDENTIAL Designator
Examples:● Read one● Read a list● Search...
CONFIDENTIAL Designator
CQRS & Event Sourcing
CONFIDENTIAL Designator
Why CQRS?
CONFIDENTIAL Designator
Performance
CONFIDENTIAL Designator
...and...
CONFIDENTIAL Designator
● Distribution● Availability● Integration● Analytics
CONFIDENTIAL Designator
WHEN CQRS?
CONFIDENTIAL Designator
Single Source of Truth: The WRITE data store
CONFIDENTIAL Designator
Next: Create your READ data stores.
CONFIDENTIAL Designator
Add Events to update data stores.
CONFIDENTIAL Designator
More about Event Sourcing:
CONFIDENTIAL Designator
It’s WRITE-ONLY
CONFIDENTIAL Designator
You get audit trails/history built in
Well … that’s nice
CONFIDENTIAL Designator
Events are immutable
CONFIDENTIAL Designator
The Event Store fires events that are independent of the origin
So convenient. I guess it’s a convenience store yuk yuk
CONFIDENTIAL Designator
Events...
CONFIDENTIAL Designator
Maintain a Materialized View
CONFIDENTIAL Designator
Integrate external systems
CONFIDENTIAL Designator
Are not reliant on a database schema
CONFIDENTIAL Designator
Can materialize the current state(This can be a batch job!)
CONFIDENTIAL Designator
Events follow the “Fire-and-Forget” model of operation.
CONFIDENTIAL Designator
It was time for another pretty picture
CONFIDENTIAL Designator
Somethings that affect your decision regarding updating READ stores:
CONFIDENTIAL Designator
Latency
CONFIDENTIAL Designator
Size
CONFIDENTIAL Designator
Staleness
CONFIDENTIAL Designator
Ownership
CONFIDENTIAL Designator
Security
CONFIDENTIAL Designator
Nature and depth of information
CONFIDENTIAL Designator
So how do we find the current state?
CONFIDENTIAL Designator
Cheat Mode = ON(It’s okay to bookmark data)
Up up down down left right left right B A
CONFIDENTIAL Designator
CQRS considerations:
CONFIDENTIAL Designator
CQRS considerations:● Complexity
CONFIDENTIAL Designator
CQRS considerations:● Complexity● Consistency
CONFIDENTIAL Designator
CQRS considerations:● Complexity● Consistency● Communication
CONFIDENTIAL Designator
Distributing events using a Message Broker
CONFIDENTIAL Designator
CONFIDENTIAL Designator
KafkaStreaming platform (with ordered delivery)
CONFIDENTIAL Designator
Kafka uses a Publish/Subscribe model with Topics
CONFIDENTIAL Designator
Vert.xReactive platform
CONFIDENTIAL Designator
CONFIDENTIAL Designator
CONFIDENTIAL Designator
CONFIDENTIAL Designator
github.com/reactica/rhte-demogithub.com/vert-x3/vertx-examples
developers.redhat.com@DonSchenck
So many URLs, so little time. What’s another browser tab open between friends?
CONFIDENTIAL Designator
This slide intentionally left blank
CONFIDENTIAL Designator
linkedin.com/company/red-hat
youtube.com/user/RedHatVideos
facebook.com/redhatinc
twitter.com/RedHat
77
Red Hat is the world’s leading provider of enterprise
open source software solutions. Award-winning
support, training, and consulting services make Red
Hat a trusted adviser to the Fortune 500.
Thank you