MongoDB on EC2 and EBS

MongoDB on Amazon Web Services

Webinar 3-14-2011

Agenda

• MongoDB deployment components• Amazon EC2 infrastructure• Deployment scenarios

MONGODB COMPONENTS

MongoDB Components

• Actual data• Needs RAM +

Disk IO

• Stores sharding configuration

• Stores small amounts of data

• Infrequently queried/updated by MongoS

• Stateless router• Typically run on App

Servers

• Can run as Arbiter• No data• Just votes to elect

primary

Replica Sets

Primary

Secondary

Secondary

Primary

Secondary

Arbiter

Primary

Secondary

Secondary

Secondary

Secondary

Sharding

MongoD

MongoD

MongoD

MongoD

MongoD

MongoD

MongoD

MongoD

MongoD

MongoD

MongoD

MongoD

ConfigD

ConfigD

ConfigD

MongoS MongoS MongoS

AMAZON EC2

Amazon EC2 Instance Types

32-bit = Don’t Use

ConfigD / Arbiter

32-bit = Don’t Use

Typical MongoD

Big MongoD

High CPU not necessary

OS• Turn off atime• Raise file descriptor limits

cat >> /etc/security/limits.conf << EOF* hard nofile 65536* soft nofile 65536EOF

• DO NOT use large VM pages• Use ext4, xfs• Use RAID

– RAID10 on MongoD– RAID1 on ConfigbDB

• Warning! Known problems with Ubuntu 10.04 & EBS– https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853– https://bugzilla.kernel.org/show_bug.cgi?id=16991

MONGODB COMPONENTS ON EC2

MongoD on EC2

MongoD

RAID 10

EBS Volumes

64-Bit EC2 Instance

LVM orMDADM

MongoS on EC2

32- or 64-Bit EC2 Instance

(Typically your app server)

MongoS

Your AppLanguage Specific Driver

Arbiter on EC2

Arbiter64-Bit EC2 Instance

(Micro Instance is Fine)

Local Instance Storage

ConfigDB on EC2

ConfigDB64-Bit EC2 Instance

(Micro Instance is Fine)

LVM or MDADM

RAID 1

EBS Volumes

DEPLOYMENT SCENARIOS

3-Node Replica Set

MongoD(Primary)

RAID 10

MongoD(Secondary)

RAID 10

MongoD(Secondary)

RAID 10

Your App

Large Large Large

2-Nodes + Arbiter

MongoD(Primary)

RAID 10

MongoD(Secondary)

RAID 10

Arbiter

Your App

Large Large Micro

US WestUS East

3-Node Replica Set Your App

MongoD(Primary)

RAID 10

Large

MongoD(Secondary)

RAID 10

Large

MongoD(Secondary)Priority = 0

RAID 10

Large

App Tier

Config DB

Shard 3Shard 2Shard 1

ShardingAmazon ELB

RUNNING IN EC2

Operational Considerations

• Automating Deployment – Puppet – Chef – Amazon CloudFormation

Backup / Restore

• EBS Snapshots– Also for rapid replica provisioning

• LVM Snapshots• Use secondaries for zero downtime backups

• http://www.mongodb.org/pages/viewpage.action?pageId=19562846

Security GroupsComponent Default PortsMongoD 27017, 28017 (Status page)MongoD Shard Server 27018MongoS 27017ConfigDB 27019

• All nodes listen on all interfaces unless –bind_ip is specified

• Specify alternate port with –port • Don’t mix names & IP’s. Names preferred• Use security groups as access control lists • (e.g. allow 27017 from app servers, but not

everybody)

Final thoughts

• EC2 is a great platform for Mongo

• Lots of production use

• Try it out! Test a big cluster for cheap!

• We’re here to help!

Questions?

• I am– Jared Rosoff ([email protected]) @forjared

• Try it out! – http://www.mongodb.org/downloads– http://www.mongodb.org/display/DOCS/Amazon

+EC2