CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Mobile Media Site using Drupal & MongoDB Presented By: Yash Badiani, Big Data Practice Lead, CIGNEX Datamatics Gaurav Khambhala, Technical Lead, CIGNEX Datamatics 30 th July, 2013
32
Embed
Create a Scalable Mobile Media Site with Drupal and MongoDB
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Mobile Media Site using Drupal & MongoDB
Presented By:Yash Badiani, Big Data Practice Lead, CIGNEX DatamaticsGaurav Khambhala, Technical Lead, CIGNEX Datamatics30th July, 2013
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
About Acquia
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Acquia – Enterprise Guide to Drupal
3
Co-founded by the Drupal project’ creator in 2007, Acquia has helped more than 2,000 clients manage
growth and scale of their digital experiences.
Acquia provides wide range of Drupal products and services, including:
Acquia Cloud Enterprise GardensAcquia Network
Drupal Training Professional Services
Drupal Commons
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
About CIGNEX Datamatics
4
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
What Does CIGNEX Datamatics Do?
5
Since 2000, CIGNEX Datamatics has implemented over 400 Open Source enterprise
solutions addressing business requirements related to Portals, Content & Big Data
Enterprise productivity through Process Engineering & Information Management Solutions
• Key Highlights– Founded in 1975
– Publicly listed in India
– Annual consolidated revenue of US$100 Million
– Fortune 500 clients
– 4,400+ employees across 22 offices in 9 countries
Strategic Alliances
7
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
About the presenters
• Yash Badiani is the Big Data Practice Lead at CIGNEX Datamatics and focuses on Big Data Technologies including MongoDB & Hadoop. He has worked extensively on large Data warehousing & Business Intelligence projects with tools such as Business Objects, Microsoft SQL Server, Microstrategy, IBM Cognos.
• Gaurav Khambhala works at CIGNEX Datamatics as Technical Lead. He is
the senior member of the PHP Practice at CIGNEX Datamatics and is involved on various technology initiatives like Big Data where he focuses on integration of PHP with NoSQL sources like MongoDB. He has a wide industry experience in software development & management in Open Source technologies such as Drupal, Moodle & Wordpress.
8
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Agenda
• The Mobile Media Use Case• Requirements and Challenges• Solution : Mobile Media site using Drupal & MongoDB• Why Drupal and MongoDB?• Demo and Solution Features• Benefits • Summary
9
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
The Mobile Explosion!
10
By 2015, at least 60% of information workers will
interact with their content applications via a mobile
device
Employees work on proposals and presentation on
mobile devices while travelling
People use digital assets (videos, images) longer on
Tablets and Mobiles compared to desktops
Based on a report by a leading IT advisory firm
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Mobile Media Use Case
• Mobile Media site includes the following features:– Store a variety of Images & associated metadata– Massively Scalable to store billions of images– Access through Mobile– Create /Edit Albums– Add Images to the Albums– Add / Edit Metadata of Images– Search Images / Albums by date, metadata, albums, etc– Social Media features – Likes, comments
11
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Requirements of Mobile Media sites
12
• Fast performance• Large user base• Concurrent CRUD• Access through various channels
• Millions of digital assets• Variety of content• Complexity of data
• Rich UI features• Social features• Mobile access• Fast search
• Elastic scaling• Cost effectiveness• Centralized storage• Ease of Maintenance
• HIGH availability• Automatic failover• User management
Velocity Volume
User experience
Scalability
Security & Availability
• Easy integration• Shorter dev cycle• Faster deployment• Ease of schema design
Flexibility & Agility
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Standard Three Layered Data Architecture
13
File System
Metadata in RDBMS
Search
Standard Three Layered Storage
Application layer
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Limitations of RDBMS
• Support limited to terabytes– No support for petabytes to
exabytes
• Manage only structured data– No support for semi-structured and
unstructured data
• RDBMS don't scale inherently– Scale up/Scale out (Load Balancing
& Replication)
• Hard to shard / partition– Large data files
• Both read / write throughput not possible
– Transactional / Analytical databases
• Specialized hardware - expensive
14
RDBMS can’t manage all dimensions of data with speed & at lower cost.
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
NoSQL is the right solution
• They are schema less• Designed to support huge data volumes
• Scalable replication and distribution mechanism– Thousands of machines distributed around the world
• Massive write performance with asynchronous inserts and updates• Designed to give high query performance• Runs on commodity hardware• Most NoSQL databases are Open Source
Usage: Spatial Data storagePopular databases: Neo4j, Bigdata
Key Value / Tulip StoreUsage: Briskly changing data and
high availabilityPopular databases: Riak, Redis,
Azure Table storage
NoSQL Databases
• 4 broad data models• 120+ variants available in the market
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Requirements of Mobile Media sites - Recap
17
• Fast performance• Large user base• Concurrent CRUD• Access through various channels
• Millions of digital assets• Variety of content• Complexity of data
• Rich UI features• Social features• Mobile access• Fast search
• Elastic scaling• Cost effectiveness• Centralized storage• Ease of maintenance
• HIGH availability• Automatic failover• User management
Velocity Volume
User experience
Scalability
Security & Availability
• Easy integration• Shorter dev cycle• Faster deployment• Ease of schema design
Flexibility & Agility
Mobile Media Site
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Drupal with MongoDB Solution
18
Themes
CoreModules
Nodes TaxonomyUserRoles
Forms &Menu
CustomModules
Workflow ForumsComments& Ratings
Tagging
PHP
Web Services
3rd party & Internal
Applications
MongoDBDriver
MongosRouting Process
Replica Set
MongoDB MongoDB
Replica Set
MongoDB MongoDB
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Why Drupal?
19
PluggableArchitecture
Data Abstraction Layer
Easy to Upgrade
Secure
Active CommunityWidely Adopted
Scalable
3rd Party Tools Integration
User Management& Permissions
HTML5 & CSS Support
Drupal Open Source CMS
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Websites using Drupal
20
Website: Whitehouse.gov Website: Data.gov.uk
Website: mtv.co.uk
Website: research.yahoo.com
Website: pdx.eduWebsite:
EndPoverty2015.org
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Why MongoDB?
21
Agile and Scalable
Full Index Support
Document Oriented Storage
ReplicationQuerying
Atomic Updates
Data Processing and aggregation
High Availability
Open Source NoSQL Solution
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Customers using MongoDB
• Centralized data management platform
• 2 billion+ documents
• 20 TB of photo metadata
• TV episodes and series
• Risk solutions auditing data
22
Source: http://www.10gen.com/customers
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Demo
23
• Media site on mobile simulator• Like & comment on an image on mobile simulator• Mobile site on web browser• Verify ‘Like’ & comment of the same image on web browser• Search images & access control