Mongo db washington dc 2014

Visualizing MongoDB Objects in Concept and Practice

https://github.com/cvitter/ikanow.mongodc2013.presentation

Washington DC 2013

Introduction

•  Do you have a MongoDB database full of BSON documents crying out for visual analysis?

Agenda

•  Visualizing Objects vs. Records •  Getting Your Data from MongoDB •  JSON 101 •  Free & Open Source Visualization Libraries

–  Google Maps JavaScript API –  D3.js

•  Other Visualization Libraries You Should Know •  Questions?

Objects vs. Records

•  Document oriented data stores support dynamic and complex schemas vs. the simple, static structures in RDBMs, e.g.:

Is There Really a Difference?

•  Yes and No

•  Yes •  Obviously, document oriented formats are different from

record oriented formats; •  Few common visualizations tools designed for

traditional record based formats support document based formats

•  No •  The basic visualizations are the same even if the data

format feeding them are different

Getting your Data from MongoDB

•  mongoexport Export data from MongoDB as JSON or CSV

•  MongoDB’s Simple REST Interface •  Read only access to data •  Start your server with the --rest option •  Simple queries:

http://127.0.0.1:28017/databaseName/collectionName/!

•  Other options: •  DrowsyDromedary, MongoDB Rest, etc.

> mongoexport --db dbname --collection collectionname --out file.json!

JSON 101

•  JSON (JavaScript Object Notation) documents are built using two types of common data structures:

•  Name/value pairs (objects) and;

•  Ordered lists of values (arrays).

•  JSON is a subset of the object literal notation of JavaScript so:

•  Converting the string representation of JSON is as easy as:

{ “string” : value }!

{ “string” : [value, value, value] }!

var jsonObj = {"numbers" : [1, 2, 3] };!var numberOne = jsonObj.numbers[0];!var numberTwo = jsonObj.numbers[1];!

var jsonObj = JSON.parse("{\"numbers\":[1,2,3]}");!

A Word About the Sample Code

•  All of the code used in this presentation is available online via GitHub at: https://github.com/cvitter/ikanow.mongodc2013.presentation "

•  The project includes code from the following Open Source Projects:

•  Bootstrap: http://twitter.github.com/bootstrap/!•  JQuery: http://jquery.com/!•  D3.js: http://d3js.org/ !

•  The data samples used are simple JSON files read in using JQuery’a .ajax method

Google Maps JavaScript API

•  Requirements: •  Get an API Key (its free*) •  Have an Internet Connection

•  Build a simple example that: 1.  Creates a place holder DIV for the map object 2.  Loads the map script 3.  Initializes the map 4.  Draws markers on the map

* Up to 25,000 map loads per day for commercial applications.

•  Use a DIV to create a placeholder for the map:

•  Load the map script:

•  Initialize the map:

Creating the Map

<div id="map_canvas" style="height:450px; width:870px;"></div>!

function loadScript() {! var script = document.createElement("script");! script.type = "text/javascript";! script.src = "http://maps.googleapis.com/maps/api/js?

!key=YOUR_API_KEY&sensor=TRUE_OR_FALSE&callback=initialize";! document.body.appendChild(script);!}!

var mapOptions = {!!zoom: 11,!!center: new google.maps.LatLng( 38.8964, -77.0262 ),!!mapTypeId: google.maps.MapTypeId.ROADMAP!

};!map = new google.maps.Map(document.getElementById('map_canvas'), !

! !mapOptions);!

•  Sample documents:

•  Add markers to the map:

Drawing Markers on the Map

for (var i=0; i < locations.length; i++) {! ! ! !!!var marker = new google.maps.Marker({!! !map: map, !!! !position: new google.maps.LatLng(!! ! !locations[i].latitude , locations[i].longitude ),!! !title: convertToTitleCase(locations[i].search_field)!!});!

}!

{...},!{!

!search_field: "adams morgan",!!country: "United States",!!country_code: "US",!

!region: "District of Columbia",!!latitude: "38.9213889",!!longitude: "-77.0425000”}!

},!{...},!

The Finished Product

•  Requires the Visualization Library:

•  Create the Heat Map Data and Layer:

Creating Heat Maps

var heatMapData = new Array();!for (var i=0; i < locations.length; i++) {!

var newRecord = {location: new !google.maps.LatLng( locations[i].geoindex.lat , ! !locations[i].geoindex.lon), weight: 1};!

heatMapData.push(newRecord);!}!

! !!var heatmap = new google.maps.visualization.HeatmapLayer({!

data: heatMapData,!dissipating: true,!radius: 10,!map: map!

});!

script.src = "http://maps.googleapis.com/maps/api/js? !key=YOUR_API_KEY&sensor=TRUE_OR_FALSE&libraries=visualiza-on &callback=initialize";!


•  D3.js (d3js.org) is: “a JavaScript library for manipulating documents based on data”

•  Convert data into visualizations in the following formats: HTML, CSS, SVG

•  Download the library or include it via:

D3.js

<script src="http://d3js.org/d3.v3.min.js"></script>!

•  Select the DIV to write the SVG image to:

•  Draw the bars (rectangles):

The (Very) Basics

var chart = d3.select("#d3_canvas").append("svg")!.attr("class", "chart")!!.attr("width", chartWidth)!!.attr("height", chartHeight);!

chart.selectAll("rect")!.data(inputData)!!.enter().append("rect")!!.attr("y", function(d, i) {return i * (lineHeight + 3); })!!.attr("width", function(d, i) ! !{ return chartWidth * (d.chance_of_rain * 0.01); })!!.attr("height", function(d) return lineHeight; });!

•  Making things scale on the chart:

•  Drawing the rules:

Adding Rules

var x = d3.scale.linear()!!.domain([0, 100])!!.range([0, chartWidth]);!

chart.selectAll("line")!.data(x.ticks(10))!!.enter().append("line")!!.attr("x1", x)!!.attr("x2", x)!!.attr("y1", 0)!!.attr("y2", chartHeight)!!.style("stroke", "#ccc");!

•  Labeling the X and Y axes:

Adding Text

chart.selectAll(".rule")!!.data(x.ticks(10))!!.enter().append("text")!!.attr("class", "rule")!!.attr("x", x)!!.attr("y", 0)!!.attr("dy", -3)!!.attr("text-anchor", "middle")!!.text(String);!

!chart.selectAll("text")!

!.data(inputData)!!.enter().append("text")!!.attr("x", 0)!!.attr("y", function(d, i) { ! !return i * (lineHeight + 3) + (lineHeight / 2); })!!.attr("dx", -3) // padding-right!!.attr("dy", ".35em") // vertical-align: middle!!.attr("text-anchor", "end") // text-align: right!!.text(function(d) { return d.date; });!


Treemaps, Bubble Charts, and More

•  D3.js can accept an array of values, objects, or a function that returns an array of values

•  Some of the D3.js visualizations allow you to pass data in a

wide variety of formats, others expect a specific format

•  The Treemap and Bubble Chart samples use a really simple JSON tree structure representing multi-level parent child relationships

Treemap

Treemaps Layout

•  D3.js features different layout packs that help you build complex charts including .treemap

•  In this example the layout pack is creating a properly sized div for each node in our JSON file as opposed to creating an SVG image var treemap = d3.layout.treemap()!

!.size([width, height])!!.sticky(true)!!.value(function(d) { return d.size; });!

!var node = div.datum(data).selectAll(".node")!

!.data(treemap.nodes)!!.enter().append("div")!!.attr("class", "node")!!.call(position)!!.style("background", function(d) { ! !return d.children ? color(d.name) : null; })!.text(function(d) { return d.children ? null : d.name; });!

Bubble Chart

Other D3js.org Examples

•  NVD3 (nvd3.org) Re-usable charts and chart components for d3.js

•  Raphael JS (raphaeljs.com) JavaScript library designed to simplify working with vector graphics and build data visualizations

•  TimelineJS (timeline.verite.co) “Beautifully crafted timelines that are easy, and intuitive to use.”

More Cool Visualization Libraries

Reminder: Get the Example Code

https://github.com/cvitter/ikanow.mongodc2013.presentation

•  All of the sample code used in this presentation is available online via GitHub at:

Thank You!

Craig Vi4er

www.ikanow.com developer.ikanow.com cvi,[email protected]

@IkanowDev

github.com/ikanow/Infinit.e

Mongo db washington dc 2014

Documents