Technical Director, 10gen
@jonnyeight [email protected] alvinonmongodb.com
Alvin Richards
#MongoDBdays
Capacity Planning: Choosing What to Deploy and When
http://hci.stanford.edu/jheer/files/zoo/ex/maps/napoleon.html
But first a story…
What are the consequences of not planning?
Capacity Planning: Why, What, When
Why?
What are the consequences of not planning?
Capacity Planning: Why, What, When
Why?
• Availability • Throughput • Latency
What?
Capacity Planning: Why, What, When
Before it's too late!
When?
Capacity Planning: Why, What, When
Start Launch Version 2
Before it's too late!
When?
Capacity Planning: Why, What, When
Start Launch Version 2
Problems
• Capacity – Under – Over – Just right?
• Prediction Models – User/Load – System(s) Behavior
• Change Velocity (reaction time) – Data/Resource-Allocation/Provisioning
Resource Usage
• Storage – IOPS – Size – Data & Loading Patterns
• Memory – Working Set
• CPU – Speed – Cores
• Network – Latency – Throughput
7,200 rpm SATA ~ 75-100 IOPS
15,000 rpm SAS ~ 175-210 IOPS
Amazon EBS/Provisioned ~ 100 IOPS "up to" 2,000 IOPS
Amazon SSD 9,000 – 120,000 IOPS
Storage Capability Example IOPs
7,200 rpm SATA ~ 75-100 IOPS
15,000 rpm SAS ~ 175-210 IOPS
Amazon EBS/Provisioned ~ 100 IOPS "up to" 2,000 IOPS
Amazon SSD 9,000 – 120,000 IOPS
Storage Capability Example IOPs
Intel X25-E (SLC) ~ 5,000 IOPS
Fusion IO ~ 135,000 IOPS
Violin Memory 6000 ~ 1,000,000 IOPS
Storage Measuring and Monitoring
Storage Measuring and Monitoring
Storage Measuring and Monitoring
Storage
• Active
• Archival
• Loading Patterns
• Integration (BI/DW)
db.blogs.stats() {
"ns" : "test.blogs", "count" : 1338330, "size" : 46915928, "avgObjSize" : 35.05557523181876, "storageSize" : 86092032, "numExtents" : 12, "nindexes" : 2, "lastExtentSize" : 20872960, "paddingFactor" : 1, "flags" : 0, "totalIndexSize" : 99860480, "indexSizes" : { "_id_" : 55877632, "name_1" : 43982848 }, "ok" : 1
}
stats()
Size of data
Average document size
Size on Disk
Size of all indexes
Size of each index
• Sorting
• Aggregation
• Connections
• Working Set – Active Data in Memory – Measured Over Periods
• Ratio to Storage
Memory
• New in 2.4 – db.serverStatus( { workingSet: 1 } )
Memory Measuring and Monitoring
MOPs
PFs
Memory and Storage MOPS: MongoDB Ops/Sec
% Disk Util
MOPS
Memory and Storage
Estimating Sample Use Case • User log on
• Read "user" collection based on supplied credentials • Update "last seen" and "ip" on "user" collection • Insert into "events" collection • Update "activity" collection for time series metrics
Estimating Sample Use Case – Worst Case
Index Read(s)
Index Write(s)
Journal Write
Data Read(s)
Data Write(s)
OpLog Write(s)
Read "user" collection based on supplied credentials
4k 4k
Update "last seen" and "ip" on "user" collection
Maybe* Maybe** sizeof(bson) 4k 4k 4k
Insert into "events" collection
4k per Index
sizeof(bson) 4k 4k
Upsert "activity" collection for time series metrics
4k 4k per index
sizeof(bson)
4k 4k 4k
* Only if page is faulted out by the O/S before this operation executes ** if indexes exists on attributes updated, index update will also occur
Estimating Sample Use Case • User log on
• Read "user" collection based on supplied credentials • 8k Read
• Update "last seen" and "ip" on "user" collection • 4k Read, 4k Write (single index), 4k Write OpLog
• Insert into "events" collection • 8k Write (single index), 4k Write OpLog
• Update "activity" collection for time series metrics • 8k Read, 8k Write (single index), 4k Write OpLog
• System load • 20k Read, 36k Write
CPU
• Non-indexed Data
• Sorting
• Aggregation – Map/Reduce – Framework
• Data – Fields – Nesting – Arrays/Embedded-Docs
MOPs
CPU
MOPs
CPU %
CPU
• Non-indexed Data
• Sorting
• Aggregation – Map/Reduce – Framework
• Data – Fields – Nesting – Arrays/Embedded-Docs
CPU
• Latency – WriteConcern – ReadPreference – Batching – Documents (and Collections)
• Throughput – Update / Write Patterns – Reads / Queries – SAN / NAS – Virtualization
Network
All of these use the same resources:
• Single Instance
• Multiple Instances (Replica Set)
• Cluster (Sharding)
• Data Centers
Deployment Types
• Storage
• Memory
• CPU
• Network
• Application Metrics
Monitoring
• MMS (MongoDB Monitoring Service)
• MongoDB: mongotop, mongostat
• Linux: iostat, vmstat, sar, etc
• Windows: Perfmon
• Load testing
Tools
• Load/Users – Response Time/TTFB
• System Performance – Peak Usage – Min Usage
Models
• Limitations – Data Movement – Allocation/Provisioning (servers/mem/disk)
• Improvement – Limit Size of Change (if you can) – Increase Frequency – Practice
Velocity of Change
• Repeat Testing
• Repeat Evaluations
• Repeat Deployment
Repeat (continuously)
• What is the working set? – How does that equate to memory – How much disk access will that require
• How efficient are the queries?
• What is the rate of data change?
• How big are the highs and lows?
Starter Questions
Technical Director, 10gen
@jonnyeight [email protected] alvinonmongodb.com
Alvin Richards
#MongoDBdays
Thank You