First app online conf

1

July 2012

Open source, high performance database

2

Today's Talk• Quick introduction to mongoDB

• Data modeling in mongoDB, queries, geospatial, updates and map reduce.

• Using a location-based app as an example

• Example works in mongoDB JS shell

3

MongoDB: scalable, high-performance NoSQL DB

4

What is mongoDB?

MongoDB is a scalable, high-performance, open source, document-oriented database.

• Fast Querying • In-place updates• Full Index Support• Replication /High Availability• Auto-Sharding• Aggregation; Map/Reduce• GridFS

5

Where can you use it?

MongoDB is Implemented in C++• Windows, Linux, Mac OS-X, Solaris

Drivers are available in many languages

10gen supported• C, C# (.Net), C++, Erlang, Haskell, Java,

JavaScript, Perl, PHP, Python, Ruby, Scala, nodejs

• Multiple community supported drivers

6

Terminology

RDBMS MongoDBTable CollectionRow(s) JSON DocumentIndex IndexPartition Shard

Join Embedding/Linking

Schema (implied Schema)

7

{ _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), author : "Asya", date : ISODate("2012-02-02T11:52:27.442Z"), text : "About MongoDB...", tags : [ "tech", "databases" ], comments : [{ author : "Fred", date : ISODate("2012-02-03T17:22:21.124Z"),

text : "Best Post Ever!" }], comment_count : 1 }

Example Document

8

BSON

•JSON has powerful, limited set of datatypes– Mongo extends datatypes with Date, Int types, Id, …

•MongoDB stores data in BSON

•BSON is a binary representation of JSON– Optimized for performance and navigational abilities– Also compression

See: bsonspec.org

9

Why use mongoDB?

• Intrinsic support for fast, iterative development

• Super low latency access to your data

• Very little CPU overhead

• No additional caching layer required

• Built in replication and horizontal scaling support

10

Building Your First MongoDB App

•Want to build an app where users can check in to a location

•Leave notes or comments about that location

11

Requirements

"As a user I want to be able to find other locations nearby"

• Need to store locations (Offices, Restaurants, etc)

– name, address, tags– coordinates– User generated content e.g. tips / notes

12

Requirements

"As a user I want to be able to 'checkin' to a location"

Checkins– User should be able to 'check in' to a location– Want to be able to generate statistics:

• Recent checkins• Popular locations

13

Collections

users

user1, user2loc1, loc2, loc3

locations checkins

checkin1, checkin2

14

> location_1 = { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021 }

Locations v1

15


> db.locations.find({name: "Din Tai Fung"})

Locations v1

16


> db.locations.ensureIndex({name: 1})> db.locations.find({name: "Din Tai Fung"})

Locations v1

17

> location_2 = { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021,

tags: ["restaurant", "dumplings"] }

Locations v2

18



> db.locations.ensureIndex({tags: 1})

Locations v2

19



> db.locations.ensureIndex({tags: 1})> db.locations.find({tags: "dumplings"})

Locations v2

20

> location_3 = { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021, tags: ["restaurant", "dumplings"],

lat_long: [52.5184, 13.387] }

Locations v3

21


lat_long: [52.5184, 13.387] }

> db.locations.ensureIndex({lat_long: "2d"})

Locations v3

22


lat_long: [52.5184, 13.387] }

> db.locations.ensureIndex({lat_long: "2d"})> db.locations.find({lat_long: {$near:[52.53, 13.4]}})

Locations v3

23

// creating your indexes:> db.locations.ensureIndex({tags: 1})> db.locations.ensureIndex({name: 1})> db.locations.ensureIndex({lat_long: "2d"})

// finding places:> db.locations.find({lat_long: {$near:[52.53, 13.4]}})

// with regular expressions:> db.locations.find({name: /^Din/})

// by tag:> db.locations.find({tag: "dumplings"})

Finding locations

24

Updating DocumentsAtomic operators:

$set, $unset, $inc, $push, $pushAll, $pull, $pullAll, $bit

25

// initial data load:> db.locations.insert(location_3)

// adding a tip with update:> db.locations.update( {name: "Din Tai Fung"}, {$push: { tips: { user: "Asya", date: "28/03/2012", tip: "The hairy crab dumplings are awesome!"} }})

Inserting locations - adding tips

26

> db.locations.findOne(){ name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021, tags: ["restaurant", "dumplings"], lat_long: [52.5184, 13.387], tips:[{ user: "Asya", date: "28/03/2012", tip: "The hairy crab dumplings are awesome!" }]

}

task - done

27

Requirements

"As a user I want to be able to 'checkin' to a location"

Checkins

– User should be able to 'check in' to a location– Want to be able to generate statistics:

• Recent checkins• Popular locations

28

> user_1 = { _id: "[email protected]", name: "Asya", twitter: "asya999", checkins: [ {location: "Din Tai Fung", ts: "28/03/2012"}, {location: "Meridian Hotel", ts: "27/03/2012"} ] }

> db.users.ensureIndex({checkins.location: 1})> db.users.find({checkins.location: "Din Tai Fung"})

Users and Checkins

29

// find all users who've checked in here:> db.users.find({"checkins.location":"Din Tai Fung"})

Simple Stats

30


// find the last 10 checkins here?> db.users.find({"checkins.location":"Din Tai Fung"}) .sort({"checkins.ts": -1}).limit(10)

Simple Stats

31


// find the last 10 checkins here: - Warning!> db.users.find({"checkins.location":"Din Tai Fung"}) .sort({"checkins.ts": -1}).limit(10)

Simple Stats

Hard to query for last 10

32

> user_2 = { _id: "[email protected]", name: "Asya", twitter: "asya999", }

> checkin_1 = { location: location_id, user: user_id, ts: "20/03/2010" }

> db.checkins.ensureIndex({user: 1})> db.checkins.find({user: user_id})

User and Checkins v2

33

// find all users who've checked in here:> location_id = db.checkins.find({"name":"Din Tai Fung"})> u_ids = db.checkins.find({location: location_id}, {_id: -1, user: 1})> users = db.users.find({_id: {$in: u_ids}})

// find the last 10 checkins here:> db.checkins.find({location: location_id}) .sort({ts: -1}).limit(10)

// count how many checked in today:> db.checkins.find({location: location_id, ts: {$gt: midnight}} ).count()

Simple Stats

34

// Find most popular locations> agg = db.checkins.aggregate( {$match: {ts: {$gt: now_minus_3_hrs}}}, {$group: {_id: "$location", numEntries: {$sum: 1}}} )

> agg.result [{"_id": "Din Tai Fung", "numEntries" : 17}]

Aggregation- in Mongo 2.1+

35

// Find most popular locations> map_func = function() { emit(this.location, 1); }

> reduce_func = function(key, values) { return Array.sum(values); }

> db.checkins.mapReduce(map_func, reduce_func, {query: {ts: {$gt: now_minus_3_hrs}}, out: "result"})

> db.result.findOne() {"_id": "Din Tai Fung", "value" : 17}

Map Reduce

36

Deployment

37

P

Deployment

• Single server- need a strong backup plan

38

Deployment


• Replica sets- High availability- Automatic failover

P

P S S

39


• Replica sets- High availability- Automatic failover

• Sharded- Horizontally scale- Auto balancing

Deployment

P S S

P S S

P

P S S

40

MongoDB good for many use cases

User Data Management High Volume Data Feeds

Content Management Operational Intelligence E-Commerce

41

Customers

42

@mongodb

conferences, appearances, and meetupshttp://www.10gen.com/events

http://bit.ly/mongofb

Facebook | Twitter | LinkedInhttp://linkd.in/joinmongo

download at mongodb.org

support, training, and this talk brought to you by

First app online conf

Technology