MongoDB Scalability Best Practices
Jason Terpko DBA, Rackspace/ObjectRocket www.linkedin.com/in/jterpko
1
Background • Started out in relational databases in public
education then financial services • Next came online media distribution combined with a
paywall • For analytics, started working with columnar
databases and engines with compression • First NoSQL project was application regression
testing • Made the switch to NoSQL at ObjectRocket by
Rackspace
www.objectrocket.com 2
Overview
• General Scalability • Schema Design • Scaling Options in MongoDB • Locking • Sharding • Right Tool For the Job
www.objectrocket.com 3
General Scalability
www.objectrocket.com
Ver(cal(Up)Horizontal(Out)
4
Schema Design • Field Types • Field Names • Arrays • Embedded Documents • Document Size
www.objectrocket.com 5
Documents
www.objectrocket.com
{"_id":ObjectId("570680d891442f6efaff2005"),"name":"JohnDoe","age":45,"height":5.9,"weight":NumberLong(165),"updated_at":"2016-01-0715:46:32","ac(ve":true,"nickname":null,"zipcodes":["10013"],"phone":"212-555-1212"
}
{"_id":ObjectId("570680d891442f6efaff2005"),"n":"JohnDoe","a":45,"h":5.9,"w":NumberLong(165),"u":ISODate("2016-01-07T15:46:32.085Z"),"ac":true,"nn":null,"z":["10013"],"p":"212-555-1212"
}
6
Scaling Options In MongoDB • Vertical
• Replica Set • Sharding (Horizontal) • Pluggable Engine
www.objectrocket.com 7
Vertical
www.objectrocket.com
Client
• Hardware/Topology• ReducedComplexity• IncreasedHardwareCost• UpgradeCycles
• Applica(onUseCases• RequiresReadConsistency• NotAbleToShard• ShardingNotRequired
8
Replica Set
www.objectrocket.com
Client
• Hardware/Topology• ReducedComplexity• ReducedHardwareCost
• Applica(onUseCases• AllowsForEventualConsistency• NotAbleToShard• ShardingNotRequired
9
Sharding
www.objectrocket.com
MongoS
• Hardware/Topology• IncreasedComplexity• BeherU(liza(on
• Applica(on• WrihenForSharding• Op(onalEventualConsistency• Increasedthroughput
10
Pluggable Engine API
www.objectrocket.com 11
MMAP
Locking • MongoDB Version • Database And Collections • Engine • Sharding
www.objectrocket.com 12
Locking
www.objectrocket.com
• MongoDBDefaults• 2.4&2.6–Database• 3.0–Collec(on• 3.2–Document
• DatabaseandCollec(onDrops• Database–Notfreebutcheap• Collec(on-Free
• Engine• MMAP–Database• MMAPv1–Collec(on• WiredTiger–Document• Rocks–Document
• TTLandSecondary's• AddedTTLDeletes• Replica(onBatching
13
Sharding • MongoS • Shard Key Selection • Distributed Writes • Targeted Operations
www.objectrocket.com 14
MongoS
www.objectrocket.com
Applica(on
Applica(on
LoadBalancer
Applica(on
15
Shard Key Selection
www.objectrocket.com
• Limita(ons• Opera(ons• UniqueConstraints• Immutable
• Process• Variety(SchemaAnalyzer)• system.profile• StagingandQA
• Chunks• Preven(ngJumbos• Pre-splipngCollec(ons• BalancingOverhead
• Throughput• DistributedWrites• TargetedOps• MergingofData
16
Non-Distributed Writes
www.objectrocket.com
10% 20% 30% 40%
Notanefficientuseofallresources.
moveChunk()Key:{u:1}
17
Distributed Writes
www.objectrocket.com
25% 25% 25% 25%
Efficientuseofallresourcesandwritescopes.
Key:{u:“hashed“}
18
Targeted Operations
www.objectrocket.com
Efficientuseofresources.
>db.mycoll.find({u:"...",c_at:"..."})
Key:{u:1,c_at:1}
19
Partially Targeted
www.objectrocket.com
Efficientuseofresources.
Key:{u:1,c_at:1}
>db.mycoll.find({u:"...",u_at:"..."})
20
Scatter-Gathers
www.objectrocket.com
Notanefficientuseofresources.
Key:{u:1,c_at:1}
>db.mycoll.find({u_at:"..."})
21
Using the Right Tool : Mongo
• Unstructured Data
• Retail Catalog Data • Social Media User Data • Analytic Data Rollups • Gaming Profiles
www.objectrocket.com 22
Using the Right Tool : ... maybe not Mongo
• Text Search • Geo • Key / Value • Transactions
www.objectrocket.com 23
Questions?
www.objectrocket.com 24
Media Resources:
MongoDB Leaf - http://bit.ly/29bbJIR RocksDB Logo - http://bit.ly/29cYFUS WT Logo - http://bit.ly/294QEhw
Thank you! Address: 401 Congress Ave Suite 1950 Austin, TX 78701 Support: 1-800-961-4454 Sales: 1-888-440-3242
www.objectrocket.com
25