MarkLogic - Managing a New Generation of Data

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Managing A New Generation of Data By Diane Burley, Chief Content Strategist, MarkLogic Corporation Finally, an agile data environment for seamless application development

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 2

Are you on offense or defense?

Which CDO Are You?

http://blogs.gartner.com/mark_mcdonald/files/2013/03/Slide21.jpg

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 4

Data is Growing at a Staggering Rate

44 ZB

8 ZB

2015 2020 Source: IDC

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 5

But Which Database?

NOSQL

NEWSQL

HADOOP

KEY VALUE STORES

DOCUMENT DATABASES

GRAPH DATABASES

IN-MEMORY DATABASES

MAPREDUCE

OBJECT ORIENTED

WIDE COLUMN STORES

SEARCH ENGINES

ANALYTICS

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 6

Enterprise IT Faces Unprecedented Challenge Leveraging Both Heterogeneous and Unstructured Data

OLTP Warehouse

Data Marts

?

Reference Data

Archives

12% Structured 88% Unstructured

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 7

Relational Databases Are Not Designed to Solve This Problem

Inability of Companies to Store, Manage, and Search Their Data

OLTP Warehouse

Data Marts

Unstructured Data

?

Reference Data

Archives

0

10

20

30

40

50

2015 2020Structured Unstructured

Source: IDC

44 ZB

8 ZBs

Explosion of Heterogeneous Data

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 8

The Endless Cycle of Data Normalization

1 2

3 4

Take snapshot of current data Build master data model based on initial view

Extract, transform, & load data into data model

Revise static model & restart process for new data

x

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 9

The Endless Cycle of Data Normalization

1 2

3 4

Take snapshot of current data Build master data model based on initial view

Extract, transform, and load data into data model

Revise static model and restart process for new data

x

2-5 years $5M++

UNDERSTANDING THE GENERATIONAL SHIFT

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 11

Any Structure Era “For all your data!” Massive scale Built for heterogeneous

and unstructured data Faster time-to-results Commodity hardware Fraction of the cost

Generational Shift in Database Market

Relational Era “For all your structured data!” Bad for unstructured Difficult for heterogeneous Proprietary hardware Expensive

Hierarchical Era “For your application “data!" Proprietary hardware Expensive

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 12

Gartner Online Transaction Processing RDBMS Magic Quadrant by Betsy Burton and Kevin H. Strange, May 2, 2002

Operational Database Market Static for Over a Decade 2013 2002

Gartner Magic Quadrant for Operational Database Management Systems by Donald Feinberg, Merv Adrian, Nick Heudecker, October 21, 2013

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 13

*Gartner Magic Quadrant for Operational Database Management Systems by Donald Feinberg, Merv Adrian, Nick Heudecker, October 16, 2014

2014: MarkLogic − Only NoSQL Vendor in Leaders Quadrant 2014

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 14

Enterprise Capability: A Corporate IT Requirement

ACID: ATOMIC, CONSISTENT, ISOLATED, DURABLE

Uncompromised Data & Transaction Resiliency

"Don't lose your data!"

SECURITY

Enterprise-grade, Fine-grained Access

"Protect your data!"

HIGH AVAILABILITY DISASTER RECOVERY

Automatic Failover, Replication, Backup/Recovery

"Prepare for the worst!"

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 15

Core Differentiator: Purpose-built for the Enterprise

RELATIONAL OPEN SOURCE

ACID TRANSACTIONS ✔ ✔ ✗

SECURITY ✔ ✔ ✗

HIGH AVAILABILITY & DISASTER RECOVERY ✔ ✔ ✗

SCHEMA-AGNOSTIC ✗ ✔ ✗

SCALE-OUT ✗ ✔ ✔

ELASTIC ✗ ✔ ✗

TIERED STORAGE ✗ ✔ ✗

SEMANTICS ✗ ✔ ✗

media

Media and Publishing (& Marketing)

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 17

Digital Supply Chain

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 18

Digital Supply Chain

fin serv

Financial Services and Banking

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 20

Front Office Systems

Workflow Components

Reporting System

Derivatives Trading Platform MarkLogic Reduces Operational Complexity & Lowers Cost

CHALLENGES The legacy Sybase system was insanely complex with 20 copies to ensure required uptime. Each new bespoke front office system necessitated expensive ETL into this complex system, and all of the reporting applications also had to be woven together. The bank needed to understand the risk in the portfolio and needed a simpler solution. REQUIREMENTS • ACID transactions to ensure no data is lost • Enterprise-level uptime with proven HA/DR • Security that meets the bank's high standards • API's and Application services to support

new reporting requirements and future application needs

insurance

Insurance

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 22

Healthcare.gov Marketplace & Data Services Hub MarkLogic Fast Time-to-Results Meets Deadlines in 12 Months

CHALLENGES Scoped to be built on Oracle, CMS quickly realized that they would not be able to meet the 2013 deadline because schema changes and data modeling would take too long. REQUIREMENTS • ACID Transactions to support

operational workload • Schema flexibility to support

changing data from payers and changing requirements from other Federal agencies

• Security to ensure private data stayed safe

• Massive scalability – 80,000 concurrent users & 170,000 request per minute

Health Insurance Payers Income and Eligibility Confirmation

State Exchanges Healthcare.gov

HIM Data Services Hub

DSH

govt

Government

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 24

Unified Government Counter-Terrorism System MarkLogic Unified Platform Supports Explosive Data Growth

CHALLENGES As intelligence data poured into various databases, they were unable to move data to various disparate systems because ETL was too costly and took too long. Unified search was necessary to save lives. REQUIREMENTS • ACID transactions to support updates to

intelligence • Schema flexibility to quickly load any and all

data • Fine grained security to ensure appropriate

access controls • Massive scalability to support an enormous

and growing system • Alerts and ability to automate workflows as

data is added

Applications for Data Analysts

PDF

XLS

Energy Life Sciences Healthcare

Transportation Retail Retail

Manufacturing Law

Enforcement Media &

Entertainment

Education

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 26

Making the Mental & Organizational Shift

Are you stuck in the cycle of data modeling and ETL while the applications and data changes faster than you can build?

Are you developing and iterating with agility to continuously add value to your organization by using all the data that's available?

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 27

Any Structure Era “For all your data!” Massive scale Built for heterogeneous

and unstructured data Faster time-to-results Commodity hardware Fraction of the cost

Generational Shift in Database Market

Relational Era “For all your structured data!” Bad for unstructured Difficult for heterogeneous Proprietary hardware Expensive

Hierarchical Era “For your application “data!" Proprietary hardware Expensive

Mainframe

Relational / SQL

NoSQL

THANK YOU [email protected] @dburley