www.techarchday.fi www.techarchday.fi Modern Data Platforms (in the age of microservices) Dominik Wagenknecht Senior Technology Architect Accenture
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
www.techarchday.fiwww.techarchday.fi
Modern Data Platforms
(in the age of microservices)
Dominik Wagenknecht
Senior Technology Architect
Accenture
www.techarchday.fi
About Dominik
Dominik Wagenknecht
Accenture Vienna / Austria
Senior Technology Architect
Emerging Technology
www.techarchday.fi
Accenture Vienna
www.techarchday.fi
SO YOU ADOPTED MICROSERVICES
Chapter 1
www.techarchday.fi
So you adopted microservices
• CI/CD pipeline is pure magic!
• Super-motivated Teams
• Business feels progress
• Great Monitoring
• Data?
www.techarchday.fi
What changed?
Browser
API Gateway
Service Service
Service Service Service
DB DB DB
Stateless
Model
API + Services!
www.techarchday.fi
What hasn’t changed?
Browser
API Gateway
Service Service
Service Service Service
DB DB DBBasically CRUD
Synchronous
www.techarchday.fi
BUSINESS WANTS MORE
Chapter 2
www.techarchday.fi
Business wants more
• More Interaction!
– Mobile Push with content
– Personalized eMail,…
• More Smartness!
– Fraud Detection
– Next-best-action
www.techarchday.fi
Mobile Push “OldSchool”
• We need to de-couple!
– Events need to reach a Queue
– Logic listens (MDB?), then push
• Issues?
– Composes really badly
– Force people to testing Queues + MDB’s
www.techarchday.fi
Data Layer is holding you back
Browser
API Gateway
Service Service
Service Service Service
DB DB DB
ServiceQueue
AppleGoogle
Data flow
async
www.techarchday.fi
MODERN DATA IN S, M & L(+ ONE DARK HORSE)
Chapter 3
www.techarchday.fi
Detour: Every op an event
• Service initiates calls
• Modern DB’s separate
– Ops first go to a (binary) log
– Then data is written into core store async
Application(s)
Database
INSERT INTO...INSERT INTO...COMMIT
(Commit)Log
Data
www.techarchday.fi
Modern Data in Small
• Idea: Don’t do it yourself
• Native
– Firebase, RethinkDB
– PipelineDB (kind of)
• Attached
– Meteor’s LiveQuery
S
www.techarchday.fi
Modern Data in Medium
• Idea: Stay lean, keep control
• Messaging
– Something simple like RabbitMQ
– Kafka if you want (but not more)
• Fill from app (or there’s an easy hook)
• Just process, add leader-election
M
www.techarchday.fi
Modern Data in Large
• Idea: Establish a big data Platform
• Really only two choices
– Hadoop with HBase, Kafka, Spark Streaming / Storm / Samza / Flink, etc…
– Cassandra (mostly used with Spark)
L
www.techarchday.fi
Detour: Hook up to Legacy
Application(s)
Database
INSERT INTO...INSERT INTO...COMMIT
(Commit)Log
Data Log Replicator e.g. Kafka
Async starts here!
www.techarchday.fi
PRAGMATISM
Chapter 4
www.techarchday.fi
Some tips
• Physical != Logical
• Stay small
• Be smart about moving parts
www.techarchday.fi
Dark Horse: In-Memory Grid
• Not “micro”, “stateless” or anything
• Think Cache + Logic + Streams
• Great (OpenSource) choice, in Java-world alone:
– Apache Ignite (GridGain)
– Apache Geode (Incubating, Pivotal Gemfire)
– Hazelcast
– …
www.techarchday.fiwww.techarchday.fi
THANKS
Related Documents
![Nadine dietz the cmo solution guide to leveraging new technology and marketing platforms at oracle modern marketing[2]](https://static.cupdf.com/doc/110x72/55ca4168bb61eb24518b48af/nadine-dietz-the-cmo-solution-guide-to-leveraging-new-technology-and-marketing.jpg)
