Geoindexing with MongoDB

Geoindexingwith

MongoDBLeszek Krupiński

WebClusters 2012

About me

On-line since 1997

Funny times

1 hr of internet for 1 USD

First social site:geocities

My first web page

What do I do now

Day-time jobManaging team of developers for Polish Air Force

Side:consulting, optimizing,

desiging

Buzzwords incoming!

The Internet2008

Web 2.0

http://en.wikipedia.org/wiki/File:Web_2.0_Map.svg CC-BY-SA-2.5

Be social in your bedroom

alone.

The Internet2012

Web 3.0

Why geospatial?

Needs shifted

Why?Because they could.

How to implement?

Database. Duh.

Keep, but also query

Is there a person at

53.438522,14.52198?

Nope.

Is there a person at

53.438522,14.52199?

Nope.

Is there a person at

53.438522,14.52199?

Yeah, here’s Johnny!

Not too useful.

Give me nearby homies.

Within the range of 1 km there is:

• Al Gore (53.438625,14.52103)• Bill Clinton (53.432531,14.55127)• Johnny Bravo

(53.438286,14.52363)

Now that’s better.

Geoindexing.Nothing new.

Oracle, PostreSQL, Lucene/Solr, even

MySQL (via extensions)

SELECT c.holding_company, c.location FROM competitor c, bank bWHERE b.site_id = 1604 AND SDO_WITHIN_DISTANCE(c.location, b.location, ’distance=2 unit=mile’) = ’TRUE’

ORACLE

SQL is so last year

Let’s use something cool

MongoDB.Because all the cool kids use NoSQL now

Why MongoDB?

Choose your NoSQL wise.

NoSQL in MongoDB

• Document –based• Queries (JS-like syntax)• JSON-like storage

Why MongoDB?Use Cases• Archiving• Event logging• Document and CMS• Gaming• High volume sites• Mobile• Operational datastore• Agile development• Real-time stats

Features• Ad hoc queries• Indexing• Replication• Load Balancing• File Storage• Aggregation• Server-side JavaScript• Capped collections

http://en.wikipedia.org/wiki/Mongodb

Back to geo.

{loc: [ 52.0, 21.0 ],name: ”Warsaw”,type: ”City”

}

db.nodes.ensureIndex({loc: '2d'})

That’s it.

Query• Exact

o db.places.find( { loc : [50,50] } )

• Nearo db.places.find( { loc : { $near : [50,50] } } )

• Limito db.places.find( { loc : { $near : [50,50] } } ).limit(20)

• Distanceo db.places.find(

{ loc : { $near : [50,50] , $maxDistance : 5 } }).limit(20)

Compound index• db.places.ensureIndex(

{ location : "2d" , category : 1 });

• db.places.find({

location : { $near : [50,50] },category : 'coffee‚

});

Bound queries• box = [

[40.73083, -73.99756],[40.741404, -73.988135]

]• db.places.find(

{"loc" : {"$within" : {"$box" : box }}

})

Problems

Units

Coordinates in arc units Distance in

kilometers

In query

earthRadius = 6378 // km

multi = earthRadius * PI / 180.0

range = 3000 // km

… maxDistance : range * multi…

In results

pointDistance =

distances[0].dis / multi

Earth is not flat.

Problem: can’t use linear distance

Earth isn’t flat too.

Solution?Use approximation.

MongoDB has it built-in

distances = db.runCommand({ geoNear : "points", near : [0, 0], spherical : true,

maxDistance : range / earthRadius /* to radians */} ).results

Focus: runCommanddistances = db.runCommand({ geoNear : "points" …

Sort by distanceOnly with runCommand

Automatically sorted• db.runCommand(

{ geoNear : "places" , near : [50,50], num : 10 });

• {"ns" : "test.places","results" : [

{ "dis" : 69.29646421910687, "obj" : … },{ "dis" : 69.29646421910687, "obj" : … },…

],…

}

Demo

OpenStreetMaps database of Poland

imported into MongoDB

14.411.552 nodes

3GB of raw XML data

PHP in virtual machine

Imported about 100.000 nodes every

10s.

Pretty cool, eh?

Kudos to Derick Rethans

Part of this talk was inspired by his talk

Questions?

Thanks!Rate me at https://joind.in/talk/view/6475

Geoindexingwith

MongoDBsupplement

Leszek KrupińskiWebClusters 2012

Why MongoDB?

Evaluate.

PostGIS is cool too.(but it’s SQL, meh)

Why MongoDB?Use Cases• Archiving• Event logging• Document and CMS• Gaming• High volume sites• Mobile• Operational datastore• Agile development• Real-time stats

Features• Ad hoc queries• Indexing• Replication• Load Balancing• File Storage• Aggregation• Server-side JavaScript• Capped collections

http://en.wikipedia.org/wiki/Mongodb

If you need other features of MongoDB,

use it

If you don’t, evaluate.

Evaluate.

Demo(hopefully)

Questions?

Please leave feedback!Rate me at https://joind.in/6475