MongoDB and MongoMK Source Event

• Click to edit Master text styles

– Second level

• Third level

– Fourth level

» Fifth level

1

MongoDB & MongoMK

Mark Puddick (MongoDB) and Yuval Ararat (Adobe)


– Second level

• Third level

– Fourth level

» Fifth level

2

• MongoDB?

• Enterprise on my mind

• Moving In!

• Practice of the Best!

Agenda


– Second level

• Third level

– Fourth level

» Fifth level

3


– Second level

• Third level

– Fourth level

» Fifth level

4

• Document Orientated Database

• Open source

• High performance

• Horizontally scalable

• Full featured

What is

MongoDB?


– Second level

• Third level

– Fourth level

» Fifth level

5

• JSON

• Binary = BSON

• Index-able

Document Model


– Second level

• Third level

– Fourth level

» Fifth level

6

• Replica Set

• Sharding

Clustering


– Second level

• Third level

– Fourth level

» Fifth level

7

• Replica Set – two or more copies

• Self-healing shard

• Addresses availability considerations

• High Availability

• Disaster Recovery

• Maintenance

• Deployment Flexibility

• Data locality to users

• Workload isolation: operational & analytics

Replica Set


– Second level

• Third level

– Fourth level

» Fifth level

8

• Three types of sharding: hash-based, range-based, tag-aware

• Increase or decrease capacity as you go

• Automatic balancing

Sharding


– Second level

• Third level

– Fourth level

» Fifth level

9

• No sharding required

– But you can still shard

• Easy to shard a later Date

– Good shard key choice essential

Sharding?


– Second level

• Third level

– Fourth level

» Fifth level

10

• Replica sets require uneven number.

Resilience

West Coast DCCentral DC

East Coast

DC

Secondarypriority = 5

Primarypriority = 10

SecondaryPriority = 5

Secondarypriority = 10

Arbiter


– Second level

• Third level

– Fourth level

» Fifth level

11

MongoDB Management

Service (MMS)

The Easiest Way to

Run MongoDB


– Second level

• Third level

– Fourth level

» Fifth level

12

What MMS can

do

Deploy

Upgrade

Scale

Continuous Backup

Point-in-Time Recovery

Performance Alerts


– Second level

• Third level

– Fourth level

» Fifth level

13

How MMS Works


– Second level

• Third level

– Fourth level

» Fifth level

14

• Deploy – any size, most topologies

• Upgrade/Downgrade – with no downtime

• Scale – add/remove shards or replicas, with no downtime

• Resize Oplog – with no downtime

• Specify users, roles, custom roles

• Provision AWS instances and optimize for MongoDB

Common Tasks,

Performed in

Minutes


– Second level

• Third level

– Fourth level

» Fifth level

15

• Monitor multiple MongoDB metrics

• Point in Time

MMS – Monitor and

Backup


– Second level

• Third level

– Fourth level

» Fifth level

16

OAK


– Second level

• Third level

– Fourth level

» Fifth level

17

• Support Big repositories

• Distributed repository

• Improved write throughput

• Support for many child nodes

• Support for many ACLs

• Built in clustering. Instead of built on top

• OSGi friendly

OAK - Goals


– Second level

• Third level

– Fourth level

» Fifth level

18

OAK Architecture


– Second level

• Third level

– Fourth level

» Fifth level

19

• MongoMK

– Built over MongoDB as persistance layer.

– Build for rapid sync between nodes

• TarMK

– Built over Tar file system

– Build for speed.

– Can be resiliant with 1.0.8 release.

Available MK

Impl


– Second level

• Third level

– Fourth level

» Fifth level

20

• Exposes Micro kernel API

• Implements a DocumentStore

– DocumentMK and DocumentNodeStore

• MVCC Approach

• GIT/SVN-inspired DAG-based versioning model [0]

[0] http://wiki.apache.org/jackrabbit/RepositoryMicroKernel?action=AttachFile&do=view&target=MicroKernel+Revision+Model.pdf

MongoMK

http://wiki.apache.org/jackrabbit/RepositoryMicroKernel?action=AttachFile&do=view&target=MicroKernel+Revision+Model.pdf


– Second level

• Third level

– Fourth level

» Fifth level

21

• In JCR 2 sessions always reflects the latest state of the repository.

• In Oak a session reflects a stable view of the repository from the time the session was

acquired.

MVCC

Counter = 1

Session 1

Session 2

Counter = 0

Counter = 0

getCounter()?

returns 0

Counter+1 = 1

Time

Counter = 1

Counter+1 = 1

JCR

Counter = 0


– Second level

• Third level

– Fourth level

» Fifth level

22

• Advantages of MVCC:

– Session cannot contain references to nodes that are not accessible anymore

– The MVCC model makes it possible to scale the repository significantly better

• Disadvantage of MVCC:

– Session may work on data that has been updated in another session.

• avoiding long-running sessions is recommended.

• If two sessions perform modifications relying on one session seeing the other session’s

changes, call Session.refresh().

– Write skew: in two non-conflicting commits on NodeStore level can lead to JCR

level constraints being violated• exception handling on saving/committing the session was required before so keep it there.

Scaling


– Second level

• Third level

– Fourth level

» Fifth level

23

• simple JSON-inspired data model:

nodes and properties

• properties are name/value pairs

• supported property types: string,

number, boolean, array

• a property value is stored and used as

an opaque, unparsed character

sequence

Data Structure


– Second level

• Third level

– Fourth level

» Fifth level

24

PRACTICE OF THE

BEST


– Second level

• Third level

– Fourth level

» Fifth level

25

• Only traffic from trusted sources can reach mongod instances

• mongod instances can only connect to trusted outputs.

• In Addition:– The “nohttpinterface” setting for mongod instances disables the “home” status page, which

would run on port 28017 by default. Disable this option for production deployments

– The REST setting for mongod enables a fully interactive administrative REST interface,

which is disabled by default. Disable this option for production deployments.

– bind_ip setting for mongod and mongos instances limits the network interfaces on which

MongoDB programs will listen for incoming connections. You can also specify a number of

interfaces by passing bind_ip a comma separated list of IP addresses.

Hardening


– Second level

• Third level

– Fourth level

» Fifth level

26

• Use MongoDB tools to create a backup of the repository db.

• Preferred sequence:

– Create replica of MongoDB primary (if not already available)

– Optional: create checkpoint on MongoDB to handle merged conflicts

– Stop replication

– Perform backup

– Restart replication

• MongoDB Managed Service can be used to create a backup.

Backup


– Second level

• Third level

– Fourth level

» Fifth level

27

• Be sure you have the latest stable release.

• Always use 64-bit builds for production.

• Linux kernel version 2.6.36 or later.

• Ext4 and XFS file systems are preffered.

• Arbiter to run on their own server.

• Tune connection pool size to avoid flooding the instance

• Consider separating data, journal, and logs to different storage devices

• On AWS it is recommended to use Ephermal SSD or SSD Backed EBS.

General

guidance


– Second level

• Third level

– Fourth level

» Fifth level

28

• SaaS options

– MongoDB Management Service

– Scout

– Server Density

• On premise options

– Nagios

– Munin

– MongoDB Management Service

Monitoring


– Second level

• Third level

– Fourth level

» Fifth level

29

• Idempotent log of operations

• Same size on all members of a replica set.

• By default it will use 5% of available disk space

• Should be at least 72hrs to a weeks of operations.

oplog


– Second level

• Third level

– Fourth level

» Fifth level

30


– Second level

• Third level

– Fourth level

» Fifth level

31

MongoDB and MongoMK Source Event

Software

shard easy

replica sets

sharding requiredbut

types of sharding

timemms monitor

mms works

bsonindexable document

usersworkload isolation