NoSQL@Mobiliar – How we started Christian Hählen
NoSQL@Mobiliar – How we started
Christian Hählen
Content
1. Background
1.1 Short introduction
1.2 Why did we start with NoSQL?
2. NoSQL@Mobiliar
3.1 Self-developped telemetry framework on classic hardware
3.2 Toolbox / Templates for usage in Docker / Kubernetes
3. Looking back
11.09.2017NoSQL@Mobiliar 2
Background
11.09.2017NoSQL@Mobiliar 3
About the company…
Swiss Mobiliar Insurance
• Own applications as well as external applications
• Operate Windows and Linux,
directly and on Docker/Kubernetes
• Observe and anticipate development
on new technologies
Mobiliar and databases
• Oracle, SQL Server, DB2
• Linux, Windows, z/OS
11.09.2017NoSQL@Mobiliar 4
… and the speaker
Christian Hählen
• 9 years working experience
• on SQL Server as data architect & database administrator
• as a database interface developer (.NET)
• 4 years working experience as System Administrator /
Head of Infrastructure and Operations
• Working at Mobiliar as database specialist / DBA for one year
• Member of PASS and co-organizer Swiss Chapter Events in Bern
• Interested in the future of
• SQL Server
• NoSQL
• Information system technologies
11.09.2017NoSQL@Mobiliar 5
History – Why did we start with NoSQL?
• NoSQL «everywhere»
• NoSQL already in the house
• With upcoming technologies like docker
RDBMS have some struggles in adapting
How to scale?
Multi-replica write
For some applications RDBMS are not the right tool
Images (BLOBs), Objects, Graphs, JSON
Transient workflow data
11.09.2017NoSQL@Mobiliar 6
• Being a DBA…
different opinions about DBA in NoSQL and agile world
developers ask if they may use NoSQL system XYZ
be ready to answer the first developer’s question
either with
"No, we don't want this in our company because …”
or with
"yes, we made a Pilot / PoC, leading to this toolbox
and templates"
History – Why did we start with NoSQL?
11.09.2017NoSQL@Mobiliar 7
NoSQL@Mobiliar
11.09.2017NoSQL@Mobiliar 8
NoSQL@Mobiliar
Performance Monitoring
• Self-developped telemetry framework (ElasticSearch, spoud)
• Operational data from applications (Prometheus)
11.09.2017NoSQL@Mobiliar 9
NoSQL@Mobiliar
Performance Monitoring
• Self-developed telemetry framework
Linked in application
Runs on every (~800) employee laptop to monitor
performance of busines application
v1: Oracle
v2: ElasticSearch - performance
v3: spoud - flexibility
11.09.2017NoSQL@Mobiliar 10
NoSQL@Mobiliar
Performance Monitoring
• Self-developed telemetry framework
Transmits data about every step, network, I/O …
90 to 100 Millions of documents per day
Last 500 days stored
For performance: index per day
Analysation / visualisation done with Kibana
11.09.2017NoSQL@Mobiliar 11
NoSQL@Mobiliar
Performance Monitoring
• Operational data from applications
Applications are developed exposing data for
Prometheus
Docker containers
currently only showing overall data
in future should expose internal data (processes etc.)
11.09.2017NoSQL@Mobiliar 12
NoSQL@Mobiliar
Usage in Docker and Kubernetes
• Change application to use MongoDB!
Why MongoDB? – We heard about it
Does data management already have a tool of choice?
• Let’s do an evaluation of NoSQL systems
Let’s define a set of systems to be supported
Prepare toolbox and guidelines
• But we need it now…
11.09.2017NoSQL@Mobiliar 13
«If you don’t have
the time to do something
right, where are you
going to find the time
to fix it?»
Stephen King
NoSQL@Mobiliar
Toolbox / Templates for usage in Docker
• Evaluation:
RDBMS vs. NoSQL, classic infrastructure
vs. VMWare vs. Container
Oracle, SQL Server, MariaDB, PostgreSQL, MongoDB,
Couchbase, Cockroach, ArangoDB
Focus on scalability, community, container size / startup
time, UI, schema / typing
Interviews with stakeholders
• Result:
Vision of the future
recommendation of RDBMS and NoSQL systems
11.09.2017NoSQL@Mobiliar 14
NoSQL@Mobiliar
Toolbox / Templates for usage in Docker
• Pilot / PoC:
Rocket.Chat
Not off the shelf
scalable NoSQL system
automated backup
performance monitoring
deployment «as one» on our platform
DevOps Project:
Developer
Container Operator
Database specialist
11.09.2017NoSQL@Mobiliar 15
• Pilot targets:
Ready-to-use template for Docker /
Kubernetes Application
AMW & Kubernetes & Docker Deployment
Using Helm chart
Persistent storage claim
Automated backup external storage
Kubernetes Job
Covered by company backup solution
Integration in our Prometheus
health / performance of DB
GUI for DevOps to have «a feeling»
Collectable by Data Management team
NoSQL@Mobiliar
Toolbox / Templates for usage in Docker
11.09.2017NoSQL@Mobiliar 17
• Pilot state:
Ready-to-use template for Docker /
Kubernetes Application
AMW & Kubernetes & Docker Deployment
Using Helm chart
Persistent storage claim
Automated backup external storage
Kubernetes Job
Covered by company backup solution
Integration in our Prometheus
health / performance of DB
GUI for DevOps to have «a feeling»
Collectable by Data Management team
NoSQL@Mobiliar
Toolbox / Templates for usage in Docker
11.09.2017NoSQL@Mobiliar 18
Looking back
11.09.2017NoSQL@Mobiliar 19
Conclusion
11.09.2017NoSQL@Mobiliar 20
• NoSQL
Might look scary to a DBA
Hot topic
Not yet on it’s peak
Conclusion
• NoSQL
Might look scary to a DBA
Hot topic
Not yet on it’s peak
You can’t hide / ignore
If you dive into it, you feel how…
11.09.2017NoSQL@Mobiliar 21
Conclusion
• NoSQL
Might look scary to a DBA
Hot topic
Not yet on it’s peak
You can’t hide / ignore
If you dive into it, you feel how
interesting
powerful
tempting
11.09.2017NoSQL@Mobiliar 22
Conclusion
• NoSQL
Might look scary to a DBA
Hot topic
Not yet on it’s peak
You can’t hide / ignore
If you dive into it, you feel how
interesting
powerful
tempting
• Implementation @ Mobiliar
Not everybody was enthusiastic at first
Toolset and guidelines are important
Pilot ist still ongoing
More to do than we first thought
Very interesting work
11.09.2017NoSQL@Mobiliar 23
Questions
Thanks for listening …
Want to learn more about our future experiences?
Follow our new blog at www.mobiliar.ch/db-blog
11.09.2017NoSQL@Mobiliar 24