Event Sourcing in einer Microservice-ArchitekturEvent Sourcing in einer Microservice-Architektur ... Event Sourcing in einer Microservice-Architektur Michael Omann. Michael Omann Architect

Event Sourcing in einer Microservice-Architektur

Ein Erfahrungsbericht aus der Werkhalle

Michael Omann Senacor Technologies AG

Event Sourcing in einer

Microservice-Architektur

Michael Omann

Michael Omann Architect bei Senacor

Softwareentwickler!

- Retail Banking,- Commercial Banking,- Investment Banking- Banking, Banking, Banking,…

Meine Branchen und Kunden:

ERP

No to NoSQL!

eCommerce Payment Solution

eCommerce Payment Solution?

Web Shop

Sockenkäufer

Server

checkout

Händler

Käufer

Banken

Vermittler

Email Notifications

Mobile TAN

SMS TAN

Photo TAN

Payments

Data Mining

Reporting

Fraud Prevention

Security

Authorization

Ticketsystem

Analysis

Dispute

Anforderungen

Audit

Business Intelligence

B2B

B2EOnline Shops

PSP

Big Data

Checkout Funnel

Mobile Commerce

Payment Gateway

Stammdatenverwaltung

Händler

Käufer

Banken

Vermittler

Email Notifications

Mobile TAN

SMS TAN

Photo TAN

Payments

Data Mining

Reporting

Fraud Prevention

Security

Authorization

Ticketsystem

Analysis

Dispute

Audit


B2B

B2EOnline Shops

PSP

Big Data

Checkout Funnel

Mobile Commerce

Payment Gateway

Zahlungssicherheit

Händler

Käufer

Banken

Vermittler

Email Notifications

Mobile TAN

SMS TAN

Photo TAN

Payments

Data Mining

Reporting

Fraud Prevention

Security

Authorization

Ticketsystem

Analysis

Dispute

Audit


B2B

B2EOnline Shops

PSP

Big Data

Checkout Funnel

Mobile Commerce

Payment Gateway

Notifications

Händler

Käufer

Banken

Vermittler

Email Notifications

Mobile TAN

SMS TAN

Photo TAN

Payments

Data Mining

Reporting

Fraud Prevention

Security

Authorization

Ticketsystem

Analysis

Dispute

Audit


B2B

B2EOnline Shops

PSP

Big Data

Checkout Funnel

Mobile Commerce

Payment Gateway


Händler

Käufer

Banken

Vermittler

Email Notifications

Mobile TAN

SMS TAN

Photo TAN

Payments

Data Mining

Reporting

Fraud Prevention

Security

Authorization

Ticketsystem

Analysis

Dispute

Audit


B2B

B2EOnline Shops

PSP

Big Data

Checkout Funnel

Mobile Commerce

Payment Gateway

Microservice-Architektur

Account

Bank

Merchant

Checkout SMS

Email

Browser

iOS

android Payments Fraud

ServerClient

Spring Boot

Account


Bank

Merchant

Customer

BankPortal

MerchantPortal

Account


Bank

Merchant

HTTP 200 Ok

Customer

BankPortal

MerchantPortal

HTTP GET /accounts/12345

Account


Bank

Merchant

Customer

BankPortal

MerchantPortal

HTTP 200 Ok

HTTP PUT /banks/555/IBAN/DE98…

HTTP 200 Ok


Account


Bank

Merchant


HTTP 200 Ok


Customer

BankPortal

MerchantPortal

HTTP 200 Ok

HTTP POST /merchants/232/shops

HTTP 201 Created

Account


Bank

Merchant


HTTP 200 Ok


Customer

BankPortal

MerchantPortal

HTTP 503 Service Unavailable

HTTP POST /merchants/232/shops

HTTP 201 Created

ZahlungsinitierungWeb Shop

Sockenkäufer

Server

checkout

Service B

Service C

Service D

Service A

Account BankMerchant

WebShop

POST /checkouts Checkout

Zahlungsinitierung


WebShop


Zahlungsinitierung

HTTP

Käuferdaten


WebShop


Zahlungsinitierung

HTTP

Käuferdaten Händlerdaten

HTTP


WebShop


Zahlungsinitierung

HTTP


HTTP

Bankdaten

HTTP


WebShop


Zahlungsinitierung

HTTP


HTTP

Bankdaten

HTTP

Authorization

Fraud

External Services

Payments

Microservices


WebShop


Zahlungsinitierung

HTTP


HTTP

Bankdaten

HTTP

Authorization

Fraud

External Services

Payments

HTTP 200 Ok

Microservices


WebShop


Zahlungsinitierung

HTTP


HTTP

Bankdaten

HTTP

Authorization

Fraud

External Services

Payments

JVM JVM JVM

JVM

CheckoutHTTP 200 Ok

Microservices


WebShop


Zahlungsinitierung

HTTP


HTTP

Bankdaten

HTTP

Authorization

Fraud

External Services

Payments

HTTP 200 Ok

Microservices

Account Merchant Bank

WebShop


Zahlungsinitierung

HTTP


HTTP

Bankdaten

HTTP

Authorization

Fraud

External Services

Payments

HTTP 200 Ok

Microservices

Service2Service Kommunikation

Merchant

Checkout


Merchant

Checkout

HTTP GET /merchants/{id}


Merchant

Checkout

HTTP GET /merchants/{id} HTTP GET /merchants/{id}/shops/{shop-id}


Merchant

Checkout

HTTP GET /merchants/{id} HTTP GET /merchants/{id}/shops/{shop-id} HTTP GET /merchants/{id}/account


Merchant

Checkout


Latenz


Merchant

Checkout


Latenz

Timeouts?Http 5xx?


Merchant

Checkout


Latenz

Timeouts?Http 5xx?

Resiliency


Merchant

Checkout


HTTP GET /merchants/{id}/checkout-information Views


Merchant

Checkout


HTTP GET /merchants/{id}/checkout-information Views

Service-spezifischeURL-Endpoints



Zahlungsinitierung

Käuferdaten Händlerdaten Bankdaten

Authorization

Fraud

External Services

Payments

HTTP 200 Ok

Microservices


WebShop


Tests?

HTTP


HTTP

Bankdaten

HTTP

Authorization

Fraud

External Services

Payments

HTTP 200 Ok

Microservices


WebShop


HTTP


HTTP

Bankdaten

HTTP

Authorization

Fraud

External Services

Payments

HTTP 5xx Internal Server Error

Tests?

Microservices

Fail!

service-e2e-test


Checkout

low availabilitylow traffic

high availabilityhigh traffic

Verfügbarkeit


Checkout

high availability high traffic

high availabilityhigh traffic

Verfügbarkeit


WebShop


:-(

HTTP


HTTP

Bankdaten

HTTP

Authorization

Fraud

External Services

Payments

HTTP 200 Ok

Event Sourcing

Event Streams

PhoneNumberAdded

AddressUpdated

AccountLimit

Updated

reduce

Account Events Event Store

PhoneNumberAdded

AddressUpdated

AccountLimit

Updated

reduce

Current State

“Account” : { “id: “0123456789” “name: “Mike” “lastName: “Magic”

“limit” : 2000 “address: {…} “phoneNumber: “+49 126 555 555 …

}


PhoneNumberAdded

AddressUpdated

AccountLimit

Updated

reduce“Account” : { “id: “0123456789” “name: “Mike” “lastName: “Magic”

“limit” : 2000 “address: {…} “phoneNumber: “+49 126 555 555 …

}


Current State

Event Store

Current State

Event Store

Current State

“Account” : {

“limit” : “high” “mobile: “+49 126 555 555”

}

Event Store

Current State

id limit age333 2000 25368 1500 34

Event Store

Current State

“Germany” : {

“totalAccounts” : 120.000 “combinedLimit: “302.340.500”

} “Austria” : {

… }


WebShop


Event Sourcing?

HTTP


HTTP

Bankdaten

HTTP

Authorization

Fraud

External Services

Payments

HTTP 200 Ok

Microservices


WebShop


Event Sourcing

Authorization

Fraud

External Services

Payments

HTTP 200 Ok

Account Events Bank Events

Shop Events

optimierte Read ModelsMicroservices


WebShop


Event Sourcing

Authorization

Fraud

External Services

Payments

HTTP 200 Ok

Account Events Bank Events

Shop Events

optimierte Read ModelsMicroservices

Implementierung?

Bus

Write ServicePUT /accounts/333/limit

Read Service A

AccountDBEvent Store

Read Service B Read Service C

UpdateLimit Id: 333 newLimit: 2000

command

“Standardansatz”

Bus


Read Service A



check/validate

CommandHandler


Event handleCommand(UpdateLimit cmd, Account account) { if (account.isLocked) throw new IllegalStateException("Account locked"); if (cmd.getLimit() > MAX_LIMIT) throw new UnsupportedOperationException("Limit exceeded") ... ... ... return new LimitUpdated(cmd.getId(), cmd.getLimit(), account.getLimit()); }

Bus


Read Service A



LimitUpdated Id: 333 newLimit: 2000 oldLimit: 1000

Event

Account<<save>>


Bus


Read Service A



EventHandler


void handleEvent(LimitUpdated evt, AccountQueryModel account) { account.setLimit(evt.getNewLimit());}

Write ServiceWrite ServicePUT /accounts/333/limit





command

Write Service


AccountDBEvent Storecheck/validate

1. Command ändert Account State + =>

Write Service

2. Ableitung eines Events

3. Speichern in AccountDB

4. Speichern in Event Store


AccountDBEvent StoreLimitUpdated

Id: 333 newLimit: 2000 oldLimit: 1000

Event

1. Command ändert Account State

diff( , ) =>

Write Service


Event

Account

PUT /accounts/333/limit

symmetrisch?

Verteilte Transaktion!

Write Service

+ =>

Read Service

+ =>

Commands vs Events


LimitUpdated Id: 333 newLimit: 2000 oldLimit: 1000

Was möchte der Client?Wie interagiert der Client mit dem Server?

Welche Zustandsänderung(en) wurde(n) durch den Command ausgelöst?

Commands vs Events

LockAccount Id: 333

LimitChanged Id: 333 newLimit: 0 oldLimit: 1000

VIPStatusRevoked Id: 333

StatusChanged Id: 333 oldStatus: active newStatus: locked

+ =>

Write Service


Event

Account


symmetrisch?

Write Service

+ =>

Read Service

+ =>

+ => :-(

Commands dürfen keine Zustandsänderung bewirken!

+ => :-(

Events leiten sich aus Commands + Current State ab

:-)+ =>

+ => :-(

Events ändern den Zustand

+ => :-)+ => :-)




command

Write Service




AccountDBEvent StoreLimitUpdated

Id: 333 newLimit: 2000 oldLimit: 1000

Event

1. Event aus Command angeleitet + =>

2. Speichern in Event Store

3. Publishen der Events zur AccountDB

4. Events ändern Current State + =>

Write ServiceWrite Service

AccountDB

Event Store

Event

Account


AccountDB als ‘Projection’ der Events

Keine verteilten Transaktionen

Write ServiceWrite Service

AccountDB

Event Store

Event

Account


Race Condition!

AccountDB als ‘Projection’ der Events

Keine verteilten Transaktionen

Write ServicePOST /accounts/12345/transactions


Race Condition

“Account” : { “id: “0123456789” “balance” : 700 “currency: EUR

}

2x

PurchaseItem Id: 333 amount: 500 currency: EUR


“Account” : { “id: “0123456789” “balance” : -500 “currency: EUR

}

“Account” : { “id: “0123456789” “balance” : 700 “currency: EUR

}

Write ServiceAccountDB

Event Store

ItemPurchased Id: 333 amount: 500 currency: EUR

POST /accounts/12345/transactions

ItemPurchased Id: 333 amount: 700 currency: EUR

Race Condition

2x



Event Storecheck/validate

Eventual Consistency

Race Condition

2x



Validierung gegen out-of-date Account

Eventual Consistency

Race Condition

2x



Current State nicht für Plausibilisierung verwenden

Race Condition

2x

AccountDB Optimierung für lesenden Zugriff

Write Service

Event Store

Read - Write

GET /*

PUT/POST/DELETE /*

für lesenden Zugriff optimiert



Write Service 3

2x



Write Service 3

2x

1. Erzeugung des current state aus Events =>



Write Service 3

2x




Write Service 3

2x




Write Service 3

2x




Write Service 3

2x


2. Sequenzielles Validieren/Plausibilisieren



Write Service 3

2x


2. Sequenzielles Validieren/Plausibilisieren

400 Bad Request

Event Sourcing im Cluster?


Event Store

Cluster

AccountDB

Node 1

Node 22x


Event Store

Cluster

Erzeugung des current state aus Events =>

Validierung auf unterschiedlichen Knoten im Cluster

AccountDB

2xNode 1

Node 2


AccountDB

Event Store

Cluster

“Account” : { “id: “0123456789” “balance” : -500 “currency: EUR

}

2xNode 1

Node 2

Cluster Sharding

Write Service

Event Store

Cluster Sharding

AccountDB

004

001

003

006005

002

009

007

008

011010

312

997

995

995

999998

996

Shard 1 Shard 2 Shard n

……..

Write Service

Event Store

Cluster Sharding mit Akka

AccountDB

Write Service

Event Store

AccountDB

ShardRegion (Actor)


Write Service

Event Store

AccountDB

ShardRegion (Actor)

POST /accounts/12345/transactions

2x

ShardCoordinator (Actor, ClusterSingleton)

PersistentActor



WebShop


:-(

HTTP


HTTP

Bankdaten

HTTP

Authorization

Fraud

External Services

Payments

HTTP 200 Ok


WebShop


Stammdaten Eventsourced

Authorization

Fraud

External Services

Payments

HTTP 200 Ok


WebShop


Event Publishing?

Authorization

Fraud

External Services

Payments

HTTP 200 Ok

??? ???

Bus

Write Service

Read Service A


Read Service B

Pub-Sub mit Bus

Bus

Write Service

Read Service A



Pub-Sub mit Bus

Bus

Write Service

Read Service A



Pub-Sub mit Bus

:-(

Bus

Write Service

Read Service A



:-(

Pub-Sub mit Bus

Enterprise DB2

MongoDB

Bus

Write Service

Read Service A



:-(

Pub-Sub mit Bus

Enterprise DB2

:-(

MongoDB

Bus

Write Service

Read Service A



:-( :-(

Pub-Sub mit Bus

Cluster?

Bus

Write Service

Read Service A



:-( :-(

Pub-Sub mit Bus

Tests?

Bus

Write Service

Read Service A



:-( :-(

Pub-Sub mit Bus

Debugging?

Queues!

Write Service

Read Service A



Point2Point mit Queues

Read Service A

Account Events


Point2Point mit Queues

Händler Events

Bank Events

Read Service A

Account Events


:-o

Händler Events

Bank Events

Read Service A

Account Events


:-o

Händler Events

Bank Events

Operations?

Poll???

Polling mit Akka

Read Service A Read Service B Read Service C

Write Service


Polling mit Akka


Write Service


Cluster Singleton (Actor)

Polling mit Akka


Write Service


Cluster Singleton (Actor)Poll

Poll

Poll


WebShop

POST /checkouts CheckoutAuthorization

Fraud

External Services

Payments

HTTP 200 Ok


WebShop

POST /checkouts CheckoutAuthorization

Fraud

External Services

Payments

HTTP 200 Ok

Poll Poll Poll

Poll for the win


Auditlog

Audit



Audit BI

Danke!

Michael Omann [email protected]

mailto:[email protected]

Fragen?



Danke!



Event Sourcing in einer Microservice-ArchitekturEvent Sourcing in einer Microservice-Architektur ... Event Sourcing in einer Microservice-Architektur Michael Omann. Michael Omann Architect

Documents