EVENT‐DRIVEN‐ ARCHITECTURE ON AWS USING AKKA
EVENT‐DRIVEN‐ARCHITECTURE ONAWS USING AKKA
Software Engineer @firstbird
Contact me: @pfeiffer_d_
DANIEL PFEIFFER
THE JOURNEY OF EVENTSthat are stored with Akka Persistenceto be then distributed via AWSto be consumed with Akka Streamwhat comes after that
THE EXAMPLE
DOMAIN DRIVEN DESIGNWe are talking about Aggregates and Events
EVENT SOURCING"Capture all changes to an application
state as a sequence of events" -Martin Fowler, 2005
CQRS"Asking a question should not changethe answer." - Bertrand Meyer, 2012
THE DIFFERENCE
PERSISTENTACTORclass TimeEntryAggregate(id: String) extends PersistentActor{ override def persistenceId: String = ??? override def receiveCommand: Receive = ??? override def receiveRecover: Receive = ???}
A PersistentActor class represents one DDDaggregate.
class TimeEntryAggregate(id: String) extends PersistentActor{ ... override def persistenceId: String = s"time-entry-$id" ...}
A PersistentActor receives commands andpersists events.
case class CreateTimeEntry( begin: DateTime, end: DateTime)
case class TimeEntryCreated( begin: DateTime, end: DateTime, sequenceNr: Long)
A PersistentActor receives commands andpersists events.
class TimeEntryAggregate(id: String) extends PersistentActor{ ... override def receiveCommand: Receive = case CreateTimeEntry(start, end) => val event = TimeEntryCreated(start,end, lastSequenceNr + 1) persist(event){e => // update internal state // here we go after the event is persisted } } ...}
A PersistentActor recovers from the journalclass TimeEntryAggregate(id: String) extends PersistentActor{ ... override def receiveRecover: Receive = { case TimeEntryCreated(start,end) => // update internal state } ...}
STORING EVENTS TO A JOURNALIS NICE, BUT ...
others may be as well interested, so we have todistribute them.
WITHIN ONE JVM WE COULD...use Akka EventStream to publish events ...
class TimeEntryActor() extends PersistentActor{ ... persist(event){e => context.system.eventStream.publish(e) } ...}
... and subscribe from interested actorsclass EventConsumer extends Actor{ override def preStart: Unit = { context.system.eventStream.subscribe(classOf[TimeEntryActor.TimeEntryCreated } override def receive: Receive = { case e: TimeEntryCreated => //do something with that event }}
HOW DO I INTERACT WITH MYAGGREGATES?
DON'T CALL ME! CALL MYOFFICE!
EXAMPLE 1
BUT WE WANT TO BE COOL
DISTRIBUTED SYSTEMS AREHARD
WE WANT TO MAKE THEMEASIER
INTRODUCE A MESSAGEBROKER, PUB/SUB ...
THE EXAMPLE ONAWS STEROIDS
TIME ENTRY SERVICE
EMAIL SERVICE
SQS FLOWSqsSource(...) .map(msg => Envelope(msg.receiptHandle, msg.body)) .via(unmarshal()) .via(process()) .runWith(ack(...))
SQS MESSAGE{ "messageId" : "", "receiptHandle" : "", "md5OfBody" : "", "body" : ""}
SNS NOTIFICATION ENVELOPE{ "Type" : "Notification", "MessageId" : "", "TopicArn" : "", "Subject" : "time_entry.approved", "Message" : "", "Timestamp" : "", "SignatureVersion" : "", "Signature" : "", "SigningCertURL" : "", "UnsubscribeURL" : ""}
CHALLENGES
EVENT SCHEMA EVOLUTIONadding a field to an event type,remove or rename field in event type,remove event type,split event into multiple smaller events.
case class TimeEntryCreated( id: UUID, begin: DateTime, end: DateTime, timeEntryUserId: UUID, userId: UUID)
gets marshalled to{ "id" : "123456", "begin" : "2016-09-01 12:00:00", "end" : "2016-09-01 12:15:00", "timeEntryUserId" : "00000000-0000-0000-0000-000000000000", "userId" : "00000000-0000-0000-0000-000000000000"}
Then something within our schema changescase class TimeEntryCreated( ... description: String ...)
and we will have fun with that one{ "id" : "123456", "begin" : "2016-09-01 12:00:00", "end" : "2016-09-01 12:15:00", "timeEntryUserId" : "00000000-0000-0000-0000-000000000000", "userId" : "00000000-0000-0000-0000-000000000000"}
{ "id" : "123456", "begin" : "2016-09-01 12:00:00", "end" : "2016-09-01 12:15:00", "timeEntryUserId" : "00000000-0000-0000-0000-000000000000", "userId" : "00000000-0000-0000-0000-000000000000"}
needs to be transformed to that{ "id" : "123456", "begin" : "2016-09-01 12:00:00", "end" : "2016-09-01 12:15:00", "timeEntryUserId" : "00000000-0000-0000-0000-000000000000", "userId" : "00000000-0000-0000-0000-000000000000", "description" : "N/A"}
class Evolution(system: ExtendedActorSystem) extends EventAdapter{} override def fromJournal(event: Any, manifest: String): EventSeq = { val es = ... //doing evolution on event EventSeq(es) } override def manifest(event: Any): String = { val version = ??? event.getClass.getSimpleName + ":" + version } override def toJournal(event: Any): Any = ??? //write to the journal}
SCALING WITH AKKA CLUSTERON AWS
or "Who am I, and where are all the others?"
JOINING A CLUSTER
gives you information about your instancehttp://169.254.169.254/latest/meta-data/instance-id
PRESERVING MESSAGE ORDER
The query side could look like that| id | ... | version || --- | --- | ------- || 1 | ... | 45 || 2 | ... | 3 || 3 | ... | 58 |
CHOOSING THE RIGHTDATASTORE
you will make mistakes so plan for it
ASYNCHRONOUS MEANS NOTIMMEDIATE
WHY DO WE REALLY WANT TODO THAT?
WE WANT TO SLEEP BETTER
WE WANT TO GET RID OF OURMONOLITH
OUR DEVELOPMENT PROCESS BENEFITS
WE CAN ALWAYS AGGREGATE OUREVENTS
IF SOMEONE STARTS TO ASKQUESTIONS...
An audit log is included in your application for free!
SUMMARY
THE FIRST PROTOTYPE IS EASY BUT TOTACKLE THE CHALLENGES NEEDS
EFFORT!
CONFIGURE YOUR SQS QUEUESPROPERLY
LET EACH SERVICE MANAGE IT'SRESOURCES ITSELF.
ONE TOPIC PER AGGREGATE
LITERATURE
https://www.infoq.com/articles/AmazonPubSubhttp://martinfowler.com/eaaDev/EventCollaboration.html
http://martinfowler.com/bliki/CQRS.htmlhttps://github.com/dpfeiffer/event-sourcing-aws-
akka-showcasehttp://www.oreilly.com/programming/free/reactive-
microservices-architecture.htmlhttp://doc.akka.io/docs/akka/snapshot/scala/persistence-
schema-evolution.html#persistence-schema-evolution-scala http://chrisloy.net/2014/05/11/akka-
cluster-ec2-autoscaling.html