RAVENDB EMBEDDED AT MASSIVE SCALES
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
RAVENDB EMBEDDED AT MASSIVE SCALES
ABOUT MERodrigo Rosauro, Software Architect Manager at RDI
• Technology passionate• Developer• 14 years of experience in software development• Working with RavenDB since 2013
ABOUT RDI
• We make restaurant automation software for QSR
• Our software runs on more than 36K restaurants
• Estimated number of individual machines: around half a million
• Processing almost USD $50K per second of customer payments
“ONE IN A MILLION IS NEXT TUESDAY”
- Gordon Letwin, Architect for DOS 4
HOW WE USED TO PERSIST DATA ON POS... and still do (on legacy modules)
LEGACY PERSISTENCY
• Flat files• Custom data format• Event sourcing persistence• Rebuild in-memory state at
every restart
PROBLEMS WITH THAT APPROACH
•No querying – at all•High complexity to store new data•No standard data format•Lack of management tools
WHY WE DECIDED TO USE A DBMS?
WHY A DBMS?
• We started a full re-architecture of the POS platform
• We are definitely not database/persistence specialists
• We wanted to remove complexity to persist data
THE SEEK FOR A DBMS
GOALS / GUIDELINES
• The new architecture is based on plug-ins, so we wanted each individual plug-in to have its own, isolated, database• Zero administration• Easy schema upgrades• Transparent replication• Unit tests
WHY WE CHOSE RAVENDB?
WHY RAVENDB?
• .NET• Embedded operation mode• In-memory mode for unit tests• Multi-tenant• Good transparent replication options
WHAT RAVENDB ALLOWED US TO DO
RAVENDB AT RDI
• Create unit tests without mocking the database• Have advanced statistics about the data persistence• Both system-wide and per plug-in
• Transparent replication• Hot failover, data distribution & real-time backups
• Transparent encryption of data at rest
THE CHALLENGES... and how we faced them
CHALLENGE #1 – MEMORY CONSTRAINTS
SPECIALIZED, OLD HARDWAREOld hardware (sometimes 10 years old) means that we have very little memory and processing power.
MEMORY CONSTRAINTS
• Many iterations with the RavenDB support team to fine-tune its memory usage
• We learned that under these constraints, caching can be evil
• In the end, the solution was to completely disable caching on RavenDB and do some cache at the application side, only for the “hot” data.
CHALLENGE #2 – ESENT
WE ARE STILL USING RAVENDB 2.5
• Our software must support Windows XPe
• .NET 4.0
• 32 bits machines
• … this means that ESENT is our only option for now
FACT: WE DON’T LIKE ESENT(at least not for our usage scenarios)
WHY?
• Any unclean shutdown may cause a completely undetermined result• Power outages / Application crashes / Task kill
• The possible results are not easy to find out during regular testing• Crappy hardware doesn’t help
• We still face sporadic full database losses with ESENT
• Copying data between OS versions is awful
• Recovering from most unclean shutdowns requires using ESENTUTL.EXE
ESENTUTL.EXE
• We also hate don’t like ESENTUTL.EXE
• Many different commands to attempt to recover from different kinds of unclean shutdowns. Some have different syntax on different OS versions
• We had to automate all that (zero maintenance, remember?)• Almost 500 lines of code
“SUCCESS IS STUMBLING FROM FAILURE TO FAILURE WITH NO LOSS OF
ENTHUSIASM”― Winston S. Churchill, Ex-prime minister of UK
THE FUTURE
THE FUTURE
• Working with Hibernating Rhinos to improve Voron to our use case• 32 bits support• Better reliability on unclean shutdowns
• We may jump straight to RavenDB 4.0 & CoreCLR• Long-term target: Around 250K individual POS nodes running
RavenDB embedded
THANK YOU VERY MUCH
Related Documents
