Unearthing 4 Hidden Gems of MongoDB 2.6 Dan Pasette VP of Core Engineering
Hidden Gems in the 2.6 Release
Nov 01, 2014
Everyone using MongoDB is familiar with the big features of the 2.6 release — text search, $out, user-defined roles, X509 authentication, etc. But what about the little guys? Our VP of Engineering, Daniel Pasette, will take you on a tour of five small but mighty features from the 2.6 release that make your MongoDB experience more productive.
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Unearthing 4 Hidden Gems of MongoDB 2.6
Dan PasetteVP of Core Engineering
MongoDB 2.6 was big.
> Power of 2 as Default Allocation Strategy> Server-side Timeouts> Query Engine Introspection> Background Indexing on Secondaries
Here’s what you missed:
Allocation. Timeouts. Introspection. Indexing.
Gem 1.Power of 2 Allocation is now on by default.
Allocation. Timeouts. Introspection. Indexing.
What happened?Before 2.6, the default record allocation used an exact-fit strategy.
Allocation. Timeouts. Introspection. Indexing.
Why does the allocation strategy matter?
Allocation. Timeouts. Introspection. Indexing.
Allocation. Timeouts. Introspection. Indexing.
Take this section of disk with some records of various sizes.
Allocation. Timeouts. Introspection. Indexing.
Documents are resized, and records are removed.
Allocation. Timeouts. Introspection. Indexing.
Space is wasted because new records cannot fit.
Allocation. Timeouts. Introspection. Indexing.
Fragmentation!
Allocation. Timeouts. Introspection. Indexing.
When document needs to move, references must be updated. That costs I/O.
The 2.6 FixBy default, round up the record size to the next power of two.
Allocation. Timeouts. Introspection. Indexing.
Allocation. Timeouts. Introspection. Indexing.
5121024
Allocation. Timeouts. Introspection. Indexing.
5121024
804 412Reduce disk frag!
We’ve made capacity planning predictable.
Allocation. Timeouts. Introspection. Indexing.
We’ve made capacity planning predictable.What about operations?
Allocation. Timeouts. Introspection. Indexing.
Gem 2.Server-side Timeouts.
Allocation. Timeouts. Introspection. Indexing.
Server
“Our collection was indexed in staging,but we forgot to index in production.”
Allocation. Timeouts. Introspection. Indexing.
ServerC
Allocation. Timeouts. Introspection. Indexing.
“Our collection was indexed in staging,but we forgot to index in production.”
ServerC
Cretry!
Allocation. Timeouts. Introspection. Indexing.
“Our collection was indexed in staging,but we forgot to index in production.”
ServerC
C
Cretry
!
retry!
Allocation. Timeouts. Introspection. Indexing.
“Our collection was indexed in staging,but we forgot to index in production.”
The 2.6 FixUse maxTimeMS to limit how long an op can run in the database.
Allocation. Timeouts. Introspection. Indexing.
We’ve stopped the op.
Allocation. Timeouts. Introspection. Indexing.
We’ve stopped the op.But why was it slow?
Allocation. Timeouts. Introspection. Indexing.
Gem 3.Get inside the Query Optimizer.
Allocation. Timeouts. Introspection. Indexing.
What happened?For 2.6, we rewrote the query execution framework — and most of the time, it just works.
Allocation. Timeouts. Introspection. Indexing.
Allocation. Timeouts. Introspection. Indexing.
Query PlannerQuery Parser
Allocation. Timeouts. Introspection. Indexing.
Query Planner Plan Cache
?
Query Parser
Allocation. Timeouts. Introspection. Indexing.
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
Miss
Query Parser
Allocation. Timeouts. Introspection. Indexing.
Query Parser
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
Miss
Plan Enumerator
Plan
Plan
Plan
Allocation. Timeouts. Introspection. Indexing.
Query Parser
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
Miss
Plan Enumerator
Plan
Plan
Plan
Multiplan Runner
Allocation. Timeouts. Introspection. Indexing.
Query Parser
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
Miss
Plan Enumerator
Plan
Plan
Plan
Multiplan Runner
Winner!
Cache
Allocation. Timeouts. Introspection. Indexing.
Query Parser
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
Miss
Plan Enumerator
Plan
Plan
Plan
Multiplan Runner
Winner!
Cache
Allocation. Timeouts. Introspection. Indexing.
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
Query Parser
The plan cache cuts out a lot of processing.
But sometimes,queries are slow.
Allocation. Timeouts. Introspection. Indexing.
Allocation. Timeouts. Introspection. Indexing.
Query Parser
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
!!!
The plan cache is cachinga suboptimal plan.
The 2.6 FixAdd logging and introspection tools to view and manipulate the cache, such as getPlanCache.
Allocation. Timeouts. Introspection. Indexing.
Now we know what’s wrong with the query.
Allocation. Timeouts. Introspection. Indexing.
Now we know what’s wrong with the query.Let’s fix it!
Allocation. Timeouts. Introspection. Indexing.
Gem 4.Background Indexing on Secondaries.
Allocation. Timeouts. Introspection. Indexing.
P
S S
Allocation. Timeouts. Introspection. Indexing.
P
S S
> db.coll.ensureIndex({ … }, { background: true })
Allocation. Timeouts. Introspection. Indexing.
CP
S S
> db.coll.ensureIndex({ … }, { background: true })
Allocation. Timeouts. Introspection. Indexing.
What happened?Before 2.6, background index builds became foreground index builds when replicated to secondaries.
Allocation. Timeouts. Introspection. Indexing.
The 2.6 FixNow, background index buildsstay in the background.
Allocation. Timeouts. Introspection. Indexing.
CP
S S
> db.coll.ensureIndex({ … }, { background: true })
Allocation. Timeouts. Introspection. Indexing.
> Power of 2 as Default Allocation Strategy> Server-side Timeouts> Query Engine Introspection> Background Indexing on Secondaries
Here’s what we covered:
Allocation. Timeouts. Introspection. Indexing.
Small gems, big impact.
Small gems, big impact.Thank you!
Related Documents
