Data Integration and Advanced Analytics for MongoDB: Blend, Enrich and Analyze Disparate Data in a Single MongoDB View

© 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-75551

Bo BorlandVP, Field Technical Sales

Blending disparate data into a single MongoDB view for analytics

Pentaho Analytics for

MongoDB



Tesla Motors Inc

$34/per share

$229/per share

January 2013 thru June 20, 2014


Tesla Motors IncCurrent month stock performance

Tesla announces it will share all patents in spirt of open source movementSource: http://www.teslamotors.com/blog/all-our-patent-are-belong-you

June 12th -Open Source

Announcement

1. Attract and motivate the world’s most talented engineers

2. Collaborate with auto-makers on a common, rapidly evolving technology platform (the charging station network)

3. Accelerate the creation and advancement of electronic vehicles (sustainable transport) to address carbon crises

10% increase


Tesla Motors IncMixed reaction to open source news

June 12th -Open Source

Announcement

June 12, 2014 - Tesla Motors Inc. to Give Away Patents -- Good or Bad for Investors?

June 15, 2014 - Electric car groups eye collaboration over charging technology


Supercharger Charging StationsOpen Now & By 2015

June 16, 2014 - Tesla Motors, Inc. Stock Jumps on News of Possible Charging Collaboration

“Tesla stock is up about 3% this morning (at the time of this writing) after Financial Times author Henry Foy reported that Tesla, Nissan, and BMW, the world's three largest manufacturers of electric vehicles, are in "keen talks" regarding a new level of collaboration on charging networks, according to sources at all three companies.”


Twitter Data

“When important news is shared on Twitter, traders and investors need to be able to access it, and validate its importance in order to incorporate that information into their decision making process,” said Jean-Paul Zammitt, head of sales and product development for the Bloomberg Professional service.

Source: http://www.bloomberg.com/now/press-releases/bloomberg-integrates-live-twitter-feeds-with-financial-platform/

http://www.bloomberg.com/now/press-releases/bloomberg-integrates-live-twitter-feeds-with-financial-platform/





Market-Moving InsightsBlending sources that can influence stocks and financial markets

Twitter Sentiment issued by corporations, executives, government officials, economist, commentators,

media outlets.

High-quality news from various news organizations and industry bloggers

Intra-day buy/sell stock quote data


• Web service api call for live intraday equity quote data• Level 1, tick-by-tick quote data for given stock and date range• Enrich the data and load to MongoDB• Relevant fields:

• Date• Time• Bid Price• Offer Price• Bid Qty• Offer Qty

Intraday Equity Quotes


• Live Twitter API search query for Telsa Motor stock• Unstructured search result is parsed• Enriched with Pentaho predictive analytics model to derive

sentiment score• Enriched dataset loaded to MongoDB• Relevant fields:

• User• Tweet text• Retweet count• Followers• Search term• Sentiment index

Twitter Sentiment Data

+1 (Positive)

0 (Neutral)

-1 (Negative)

Advanced Analytics

Data Science Pack: Pentaho 5.1 capability with a text classification predictive model to score each tweet and load the tweets and scores into MongoDB.


Market-Moving Insights

• Intra-day buy/sell stock quote data provided by a web service

• Real time Twitter data provided by Twitter API

• Pentaho Data Integration to blend disparate data sources

• Pentaho Predictive Analytics to enrich data with twitter sentiment score

Single MongoDB view with Pentaho Analysis to make

stock investment decisions


Analyzer for MongoDB

Released Today as part of Pentaho 5.1 Release

Analyze and visualize MongoDB data thru a drag-n-drop interface.

• Native OLAP analysis on real-time data in MongoDB

• Easy to use, award-winning OLAP interface for building visualizations that can be added to dashboards

• Native integration with MongoDB leveraging the power of the MongoDB query language and aggregation framework.

Analyze blended June data to make an investment decision on Tesla Motors?


Analyzer Demonstration


Market-Moving InsightsBlending sources that can influence stocks and financial markets

Twitter Sentiment issued by corporations, executives, government officials, economist, commentators,

media outlets.

High-quality news from various news organizations and industry bloggers

Intra-day buy/sell stock quote data

BlendEnrichPredictLoad

QueryAggregate

NotifyInform


mongoDBclusterPDI ETL

Analytics

Broad ConnectivityBroad connectivity combined with powerful data integration


Concept – Data Transformations

INPUT(S) – PROCESS(ES) – OUTPUT(S)


Data Integration Demonstration


Pentaho Visual DevelopmentEliminates the Need for Complex Coding

Would you rather do this?

Scheduling

Modeling

Ingestion / Manipulation / Integration

… or this?


Equity Data Ingestion



Twitter API Data Ingestion



Pentaho Job Automates Data Load Sequence

START – CHECK – WATCH – EXECUTE – NOTIFY - FINISH


Equity Quotes SchemaEliminates the need for table joins

RDBMS Star Schema

(6-9 tables)

Mongo Hierarchy Schema

(1 collection)


Entity 360 – NoSQL ArchitectureA Blended View to Drive Revenue Growth and Service Improvements

MongoDB

CRM System

Claims

Admin. Info

PDI

Online Transactions

Documents & Images

PDI

Advanced Analytics

Business Analysis

Analyzer for MongoDB

Data Science

Pack

Aggregation

Framework

Data Science

Pack


Pentaho Analytics for MongoDB

© 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-755525 © 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-755525

OPERATIONAL DATA BIG DATA DATA STREAMPUBLIC/PRIVATE CLOUDS

DBA/ETL/BI DEVELOPER BUSINESS USER BUSINESS/DATA ANALYST

Data Integration +

Analytics + Predictive

Extensive community of

developers

Pluggable, extensible, java

architecture

Single, Open Platform for Analytics


Powerful Technologies

Online / Offline Data

For creating a single view in MongoDB


Bo BorlandVP, Field Technical [email protected]

Thank You

Data Integration and Advanced Analytics for MongoDB: Blend, Enrich and Analyze Disparate Data in a Single MongoDB View

Technology

industry bloggers

mongodb relevant

government

rights reserved

influence

media outlets

tesla motors

news organizations