Event Sourcing & CQRS

Event Sourcing& CQRSRetour d'expérience

Maher TEBOURBI@mtebourbi

Etienne Neveu@eneveu Sfeir Plays

Thursday, June 27, 13



Urban Power






MDM ServerRest API

MySql

Cassandra

AMQP

Tomcat

UI

Adapters


MySQL Cassandra

RawDatapointValidation Estimation

AggregationPersistence CEP

Context


MDM Server

Rest API

Cassandra

Netty

UI

Adapters


InputBuffer

OutputBuffer

Business Logic

Event

Business Processor


Producer

1

2

3 4

5

6

7

8

910

11

12

Handler 1

Handler n

Handler 2

Consumer


Log

Input

Pub

Response

Commit

Business Processor

Replicator

Timer

Input Ring Output Ring


InputBuffer

OutputBuffer

Business Logic

Event

Business Processor

Évènements d'entrée • Domain Model• Logique métier• Consumers

Évènements de sortie:- persistence- appels WS externes

Business Processor - Intro


@Immutablepublic class RawDatapointEvent {

private final String sensorId; private final DateTime measureTime; private final double value;

public RawDatapointEvent(String sensorId, DateTime measureTime, double value) { this.sensorId = sensorId; this.measureTime = measureTime; this.value = value; }

public String getSensorId() { return sensorId; } public DateTime getMeasureTime() { return measureTime; } public double getValue() { return value; }

}

Business Processor - Évènement d'entrée


public class RawDatapointEventConsumer implements EventConsumer<RawDatapointEvent> {

@Inject DatapointPreprocessingService preprocessingService;

@Override public void consume(RawDatapointEvent rawEvent, Model model, EventProcessor eventProcessor) { PreprocessingResult result = preprocessingService.preprocess(model, rawEvent);

if (result.hasError()) { eventProcessor.sendReply(new FailedDatapointResult(result.getErrorMsg())); }

eventProcessor.sendReply(new SuccessfulDatapointResult()); eventProcessor.process( new AddDatapointToTimeSeriesEvent(result.getAccount(), result.getSensor().getTimeSeries(), result.getDatapoint() ) ); }

}

Business Processor - Consumer


@Immutablepublic class RawDatapointEvent {

// fields, constructor, getters...

public interface Result {}

public static class SuccessfulDatapointResult implements Result {}

public static class FailedDatapointResult implements Result { private final String errorMessage; public FailedDatapointResult(String errorMessage) { this.errorMessage = errorMessage; } public String getErrorMessage() { return errorMessage; }

}

}

Business Processor - Évènement de résultat


RawDatapointEvent rawEvent = createRawEventFromJson(httpRequest.getContent()); ListenableFuture<RawDatapointEvent.Result> future = client.sendEvent(rawEvent); Futures.addCallback(future, new FutureCallback<RawDatapointEvent.Result>() { @Override public void onSuccess(RawDatapointEvent.Result result) { ... } @Override public void onFailure(Throwable t) { ... }});

Business Processor - Code client


synchronized

ConcurrentHashMap

volatile

ReentrantLock

Mono threading - Synchronisation


Mono threading - I/O


In-Memory Model - Persistence and caching


In-Memory Model - DDD


In-Memory Model - Data structures

ArrayList

HashMap

HashSet

BiMapMultimap

ConcurrentHashMap

CopyOnWriteArrayList

ReentrantLock


@Immutablepublic final class Money { // I swear, those are final! private BigDecimal amount; private Currency currency;

// For Hibernate only! Money() {}

public Money(BigDecimal amount, Currency currency) { this.amount = amount; this.currency = currency; }

public BigDecimal getAmount() { return amount; } public Currency getCurrency() { return currency; }

// For Hibernate only! void setAmount(BigDecimal amount) { this.amount = amount; } void setCurrency(Currency currency) { this.currency = currency; }

}

In-Memory Model - Immutability


@Immutablepublic final class Money { private final BigDecimal amount; private final Currency currency; public Money(BigDecimal amount, Currency currency) { this.amount = amount; this.currency = currency; } public BigDecimal getAmount() { return amount; } public Currency getCurrency() { return currency; } }

In-Memory Model - Immutability


public class UserManager { private final Map<String, User> users = Maps.newHashMap(); public void add(@Nonnull User user) { users.put(user.primaryKey(), user); } public void remove(@Nonnull String userId) { User removedUser = users.remove(userId); if (removedUser == null) throw new UserNotFoundException(userId); } public User getById(@Nonnull String userId) { User user = users.get(userId); if (user == null) throw new UserNotFoundException(userId); return user; } public List<User> getAll() { return ImmutableList.copyOf(users.values()); } }

In-Memory Model - Managers


http://commons.wikimedia.org/wiki/File:Hard_disk_Western_Digital_WD740_1_(dark1).jpg

In-Memory Model - Memory Size




Snapshots - Intro


Snapshots - Sérialisation

Protocol Buffers


Snapshots - Migration

SnapshotV4

SnapshotV5

MessagesProtobuf

V5

MessagesProtobuf

V4


Data

BV1

BV2

BV3

RTV2

RTV3Event Source Server

Speed Layer

Batch Layer

EventV2



Event Sourcing & CQRS

Technology