We broke up with the monolith, and started dating #eventSourcing - #symfonyCat

We broke up with the Monolith, and started dating Event SourcingMigrations, Event Sourcing, AWS, and other Demeter things

@JavierCane#symfonyCat - Symfony Catalunya 2016

@JavierCane

Saliendo del monolito, al Rico EventitoMigraciones, Event Sourcing, y otras cosas del meter

#symfonyCat - Symfony Catalunya 2016

Welcome!I’m Javier Ferrer González

@JavierCane

Greetings to Sergi!@SergiGP

Welcome!

Contents

1. Starting point 2. Needs 3. Migration action plan 4. Sync flow example 5. Error handling 6. Issues found 7. Conclusions

1. Starting point

Context

Mobile first product

Starting point Mobile first app

! In-house REST API

◕ symfony/symfony

◕ friendsofsymfony/rest-bundle

◕ jms/serializer

◕ nelmio/api-doc-bundle

Massive growth

Starting point Where do we come

! Startup with less than 2 years

◕ Externalize services (Parse, Kahuna) ! Funding: $200M

◕ Ads on TV in USA and Turkey

Starting point Where do we go

! Microservices architecture ! Event Sourcing ! Orchestration Layer

2. Needs

New chat system!

Needs New chat system!

! From:

◕ REST API

◕ “Chat” (messaging system, polling) ! To:

◕ WebSockets

◕ Chat (real time, push)

◕ Service isolated from the main API

◕ Backwards Compatible (mobile!)

◕ Scale! ! QA tested, Coming to a theater near you!

Needs New chat system!

! To:

◕ WebSockets

◕ Chat (real time, push)

◕ Service isolated from the main API

◕ Backwards Compatible (mobile!)

◗ Not loosing anyone

◗ Enables a “soft migration”

◕ Scale! ! QA tested, Coming to a theater near you!

3. Migration action plan

Maintaining BC

Migration action plan Maintaining BC

! Develop new chat with Scala and Akka, and WS ! Maintain BC taking advantage of Event Sourcing



! Why not share DB?

◕ Limit scheme modifications for the new chat

◕ Chaos

◕ Coupling



! SQS: Amazon Simple Queue Service

◕ Managed message queuing service

◕ Transmit any volume of data, at any level of throughput, without losing messages or requiring other services to be always available

! Why not sync through SQS?

◕ Coupling

◗ 2 subscribers => publish to 2 SQS



! SNS: Amazon Simple Notification Service

◕ Send push messages to subscribers (no polling)

◕ Scale as your needs grow ! Why sync through SNS?

◕ Decoupled

◗ N subscribers => publish to 1 SNS

◗ Open-Closed Principle compliant at a services level


! 1 SNS per microservice

◕ SNS per event: Too much detail. New event => New SNS

◕ Unique SNS: Not enough detail. Too much noise on the subscribed SQS


Sync

conas asequence


! Delete all messages coming from the legacy chat on the SQS until day X

! Migrate all conversations and messages from day 0 to X using a script

! Start consuming the SQS

4. Sync flow example

Maintaining BC

Events anatomy

! 301 @dsamblas

Sync flow example From the new chat to the legacy one

{ "data": { "id": uuid, "type": "conversation_archived", "attributes": { "conversation_id": uuid, "talker_id": uuid, "archived_at": timestamp, }, "meta" : { "created_at": timestamp, "from_legacy": bool }

First level: “protocol” data


"data": { "id": uuid, "type": "conversation_archived", "attributes": { "conversation_id": uuid, "talker_id": uuid, "archived_at": timestamp, }, "meta" : { "created_at": timestamp, "from_legacy": bool } }}

Domain aggregate data


"id": uuid, "type": "conversation_archived", "attributes": { "conversation_id": uuid, "talker_id": uuid, "archived_at": timestamp, }, "meta" : { "created_at": timestamp, "from_legacy": bool } }}

Event metadata

From the new chat to the legacy one







From the legacy chat to the new one

Sync flow example From the legacy chat to the new one









5. Error Handling

Order not guaranteed

Error handling Order not guaranteed



Conversation does not exists!

Eventuallyconsistent

Conv. archivedConv. archived

Get items

Conv. archived

Archive conv.


Conv. createdConv. created

Conv. created


Get items

Conv. archived

Archive conv.



Conv. createdLets queue it with ‘++$attempts;’


Get items

Conv. archived

Archive conv.



Conv. created

Create conv.Conv. archived’

Get items

Conv. archived’ Archive conv.

! If we find an error processing an event =>

◕ Increment its processing attempts and queue it again ! If the number of processing attempts is greater than 5 =>

◕ Queue it into a dead letter SQS ! Symfony command in order to reprocess the dead letter


SQS: Message duplication

Let itcrash

Identity generation

Error handling Identity generation

Conv. created

Message sentConv. created

Message sentGet items

Conv. created

Message sent

Send message


Conv. created



Conv. created

Message sent

Send message

Conversation does not exists! Lets create it with ID ‘b’!


Conv. created



Conv. created

Message sent

Create conv.

Send message

Conversation does not exists! Lets create it with ID ‘b’!

Lets create the conversation ‘a’!

“ Generating identifiers from the inside is a side effect. Side effects difficult testing.

“ If you generate IDs from the inside, you are not being SRP compliant.

6. Issues found

Issues found SQS -> SNS

! Order not guaranteed & events duplication ! Alternatives:

◕ RabbitMQ

◗ Why not: go fast & don’t worry about clusters

◕ Kafka

◗ Why not: Same as RabbitMQ

◗ BTW: Order guaranteed, no duplication, partitions

◕ Kinesis

◗ Why not: More expensive

◗ BTW: Order guaranteed, no duplication, partitions

◕ Furthermore: Do not add more uncertainty

Issues found Initial data migration

! Source DB node: Slave outside the production LB ! Destination DB node: Increase AWS provisioned IOPS

#basuritas

Issues found Not following SRP

! REST API

◕ Send message endpoint => create conversation if not exists. #basuritas

◕ GET conversation messages => mark messages as read

◗ Check if a conversation exists, new endpoint: ”HEAD /conversation/{id}”

7. Conclusions

Conclusions

! Event Sourcing ~~ DIP at a services level

◕ Dependency Inversion Principle (CodelyTV video)

◕ Allow OCP: One SNS publishes to many SQS

◕ Encourage SRP

◕ Decoupleeeeee

http://codely.tv/screencasts/solid-principio-inversion-dependencias/

Conclusions

! Event Sourcing ~~ DIP at a services level

◕ Dependency Inversion Principle (CodelyTV video)

◕ Allow OCP: One SNS publishes to many SQS

◕ Encourage SRP

◕ Decoupleeeeee

http://codely.tv/screencasts/solid-principio-inversion-dependencias/

Conclusions

! Isolated services

◕ Different technologies

◕ Different contexts

◕ Change context -> Mental challenge

! From PHP to Scala: 🙃 🙂 🙃 💃 🙃 😀 😐 😳 🤔 🙂 🙃 🙂 …

! Know more at #SCBCN16! (October 1st & 2nd in BCN) ! A lot of work, if you want to deal with that, we're hiring!

http://jobs.letgo.com/

Conclusions

References / To take away

! A Series of Fortunate Events ! The anatomy of Domain Event ! 6 Code Smells with your CQRS Events and How to Avoid Them ! json:api standard ! Sending Amazon SNS Messages to Amazon SQS Queues ! Subscribing a Queue to an Amazon SNS Topic

http://www.slideshare.net/matthiasnoback/a-series-of-fortunate-events-symfony-camp-sweden-2014

http://blog.arkency.com/2016/05/the-anatomy-of-domain-event/

http://danielwhittaker.me/2014/10/18/6-code-smells-cqrs-events-avoid/

http://jsonapi.org/

http://docs.aws.amazon.com/sns/latest/dg/SendMessageToSQS.html

http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqssubscribe.html

Questions?

Thanks!Contact @JavierCane || @SergiGP [email protected] Codely.TV

Credits

! Presentation base template by SlidesCarnival ! Graphics generated using draw.io ! Icon set Small-n-flat ! A Series of Fortunate Events

http://www.slidescarnival.com/

http://draw.io

http://paomedia.github.io/small-n-flat/

http://www.slideshare.net/matthiasnoback/a-series-of-fortunate-events-symfony-camp-sweden-2014

We broke up with the monolith, and started dating #eventSourcing - #symfonyCat

Technology