Transcript
8 learnings from building a Data Platform
DIGITAL INTERNATIONAL
Goal
Of this talk
About
The org - me
Why
Did we embark on this anyway?
First, take a step back
Vision
Mission
Objectives
Key Results
We have a dream: Vision
It's our vision to be the engine of customer obsessed
intelligence, continuously driving agile fact based decision making
Tangible: Objectives
We have a flat file data lake updated in daily batch with at least 1 data source
Tangible: Key results
Tables ABC are from source X offloaded in landing zone
Staging, transformation and harmonisation are automated
1 2
Learning #1
Laser focus
Learn by trying
Learn by testing
Set criteria BEFORE
Maintainability Scalability Reliability Security Cost(ability?)
etc...
Learning #2
Do POCs
Are you ready to roll?
Right skills Budget Data access Infra freedom Sponsor
Learning #3
Remove dependencies
Hindsight engineering = Pain
GDPR
Learning #4
Actively involve privsec
Design Principles
● PaaS● Automate all things● Separate compute from storage● Event driven● Loosely coupled
Learning #5
Acceptance criteria
With the business for the business
Learning #6
Include your customer
Don’t make it a big project
Don’t make it perfect
Your goal
Learning #7
Be Agile - adopt
Data is an intangible asset - make it real
Learning #8
Share!
Learnings
- Learning 1: laser focus
- Learning 2: do POCs
- Learning 3: be independent
- Learning 4: involve privsec
- Learning 5: design principles
- Learning 6: include your customer
- Learning 7: be Agile
- Learning 8: share!
Thank you
top related