OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds

NoSQL & MySQL: The Best of Both Worlds

Andrew Morgan (@andrewmorgan)www.clusterdb.com

Principal MySQL Product Manager

11th June 2014

Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.2

Safe Harbour Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


MySQL Customers

World’s Most Popular Open Source Database

WebSaaS, Hosting

Enterprise OEM / ISV’s

Telecom

http://images.google.com/imgres?imgurl=http://www.infinitytechnology.com/uploadedImages/Home/TechFocus_News/AlcatelLucent_Hor_2col_lrg.gif&imgrefurl=http://www.infinitytechnology.com/phonesystems.aspx&h=251&w=720&sz=11&hl=en&start=4&sig2=urYvX47BOQRqvADY_UMA0w&um=1&tbnid=pA91nzVKrUFImM:&tbnh=49&tbnw=140&ei=94SgSOaRCaqkpATZu-jgBw&prev=/images?q=alcatel+lucent&um=1&hl=en&suggon=0&rls=com.microsoft:en-us&sa=N


















SQL NoSQL


Session Agenda

NoSQL – What are people looking for? RDBMS – What advantages do they still have? How MySQL Delivers the Best of Both Worlds

– MySQL Cluster NoSQL attributes: Scale-out, performance, ease-of-use, schema

flexibility, on-line operations NoSQL APIs

– Key-Value store access to InnoDB (Memcached)


Types of NoSQL stores

Key-Value

• Cassandra• Memcached• BigTable• Hadoop• Voldermort

Document

• MongoDB• CouchDB

Graph

• Neo4J• FlockDB


Key-Value Store


Document Store


Graph Database


What NoSQL must deliver

Massive scalability– No application-level sharding

Performance High Availability/Fault Tolerance Ease of use

– Simple operations/administration– Simple APIs– Quickly evolve application &

schema

ScalabilityPerformance

HAEase of use



Still a role for the SQL (RDBMS)?

NoSQL

Simple access patterns

Compromise on consistency for performance

Ad-hoc data format

Simple operation

SQL

Complex queries with joins

ACID transactions

Well defined schemas

Rich set of tools

No best single solution fits all

Mix and matchScalability

PerformanceHA

Ease of useSQL/Joins

ACID Transactions


The Temptation to Jettison Relational Model Relational Model:

– Data stored in multiple tables

– Many lookups to retrieve a user’s timeline

– Is it worth the effort in setting up this complex data model?

Simpler just to store as one document?

The allure of document stores

Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy

user

friend

post

comment

like

liker

commenter

many

many

many

one

one

many

https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy




The Temptation to Jettison Relational Model

Document Model:– Entire timeline in a single

document (row)– Single lookup to retrieve the

user’s timeline– Brilliantly efficient model when

the document truly contains self-contained information

Like a real-world document!



{name: ‘Joe’, url: ‘…’ stream:[ {friend:{ name: ‘Jane’, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: ‘Joe’}, {…} ], likes: [‘Joe’, ‘Fred’, ‘Billy’] }, {…},{…},… ] }, {…},{…},…}







These are all people who have their own data that users will want to view or click through:

– Name– url– Timeline

Easy to represent with FKs in a relational model

But when the data isn’t self contained…


user

friend

post

comment

like

liker

commenter

many

many

many

one

one

many






Do I store all data for all friends and likers again at every point they appear in the document?

– Massive amount of repeated data

Wasted space Needs to be kept in sync (and

you don’t have transactions to provide consistency



{name: ‘Joe’, url: ‘…’ stream:[ {friend:{ name: ‘Jane’, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: ‘Joe’}, {…} ], likes: [‘Joe’, ‘Fred’, ‘Billy’] }, {…},{…},… ] }, {…},{…},…}






The Temptation to Jettison Relational ModelThe allure of document stores


{name: 83746251, url: ‘…’ stream:[ {friend:{ name: 9384726153, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: 83746251}, {…} ], likes: [83746251, 750730283, 2938493820] }, {…},{…},… ] }, {…},{…},…}

The reality is that the developer will store the user-ids instead

Developer is responsible for implementing ‘joins’ in the application

– e.g. scanning all timelines for where a user made a comment or liked a post

No transactions to ensure consistency between documents






MySQL Cluster: Overview• Auto-Sharding, Multi-Master• ACID Compliant, OLTP + Real-Time Analytics

HIGH SCALE, READS + WRITES

• Shared nothing, no Single Point of Failure• Self Healing + On-Line Operations

99.999% AVAILABILITY

• Open Source + Commercial Editions• Commodity hardware + Management, Monitoring ToolsLOW TCO

• Key/Value + Complex, Relational Queries• SQL + Memcached + JavaScript + Java + JPA + HTTP/REST & C++SQL + NoSQL

• In-Memory Optimization + Disk-Data• Predictable Low-Latency, Bounded Access TimeREAL-TIME


Who’s Using MySQL Cluster?


MySQL Cluster Architecture

MySQL Cluster Data Nodes

Data Layer

Clients

Application Layer

Management




Data Layer

Clients

Application Layer

ManagementManagement





Data Layer

Application Layer

ManagementManagement

Clients



On-line Operations

Scale the cluster (add & remove nodes on-line) Repartition tables Upgrade / patch servers & OS Upgrade / patch MySQL Cluster Back-Up Evolve the schema on-line, in real-time




Scale-Out Reads & Writes on Commodity Hardware

8 x Commodity Intel Servers– 2 x 6-core processors 2.93GHz – x5670 processors (24 threads)– 48GB RAM

Infiniband networking flexAsynch benchmark (NDB API)

2 4 80

200

400

600

800

1,000

1,200SELECT Queries per Minute

Number of Data Nodes

Mill

ions

4 80

204060

80100

120UPDATE Queries per Minute

Number of Data Nodes

Mill

ions


JOIN Performance

33K rows over 11 tables

Must Analyze tables for best results

mysql> ANALYZE TABLE <tab-name>;

MySQL Cluster 7.1 MySQL Cluster 7.20

102030405060708090

100

Query Execution Time Seconds

Scalability aPerformance a

HA aEase of useSQL/Joins a

ACID Transactions

a

70xMore

performance


MySQL Cluster 7.4

Performance gain over 7.3– 47% (Read-Only)– 38% (Read-Write)

Better performance and operational simplicitylabs.mysql.com

Faster node restarts– Recovering nodes rejoin the cluster

faster



Creating & running your first ClusterThe traditional way (pre-MCM & Auto-Installer)

Download & Extract•edelivery.oracle.com

•www.mysql.com•dev.mysql.com

Configure•Cluster-wide “config.ini”

•Per-mysqld “my.cnf”

Start processes•Management Nodes•Data Nodes•MySQL Servers



HA aEase of use aSQL/Joins a

ACID Transactions

a



MySQL Cluster 7.3: Auto-Installer

Fast configuration Auto-discovery Workload optimized Repeatable best practices

Specify Workload

Auto-Discover

Define TopologyDeploy



MySQL Cluster Manager

1. Download MCM/Cluster package from edelivery.oracle.com:2. Unzip3. Run agent, define, create & start Cluster!

$> bin\mcmd --bootstrapMySQL Cluster Manager 1.1.2 started

Connect to MySQL Cluster Manager by running "D:\Andrew\Documents\MySQL\mcm\bin\mcm" -a NOVA:1862

Configuring default cluster 'mycluster'...

Starting default cluster 'mycluster'...

Cluster 'mycluster' started successfully

ndb_mgmd NOVA:1186

ndbd NOVA

ndbd NOVA

mysqld NOVA:3306

mysqld NOVA:3307

ndbapi *

Connect to the database by running "D:\Andrew\Documents\MySQL\mcm\cluster\bin\mysql" -h NOVA -P 3306 -u root


MCM: Upgrade Cluster

mysql> upgrade cluster--package=7.3 mycluster;



HA aEase of use aSQL/Joins a

ACID Transactions

a




NoSQL Access to MySQL Cluster data

ClusterJMySQL

JDBC

Apps

JPA

JNIPython Ruby

ClusterJPA

Apps Apps Apps Apps Apps

Node.js

JS

Apps

mod-ndb

Apache

Apps

ndb-eng

Memcached

Apps Apps

NDB API (C++)


Apps

PHP PERL

Apps


Schema-Free apps Rapid application evolution

– New types of data constantly added

– No time to get schema extended

– Missing skills to extend schema

– Initially roll out to just a few users

– Constantly adding to live system


Cluster & Memcached – Schema-Free

<town:maidenhead,SL6>

key value

<town:maidenhead,SL6>key value

Key Valuetown:maidenhead SL6

generic table

Application view

SQL view


Cluster & Memcached - Configured Schema

<town:maidenhead,SL6>

prefix key value

<town:maidenhead,SL6>key value

Prefix Table Key-col Val-col policytown: map.zip town code cluste

rConfig tables

town ... code ...

maidenhead ... SL6 ...

map.zip

Application view

SQL view


MySQL 5.6 Memcached with InnoDB

8 32 128 5120

10000

20000

30000

40000

50000

60000

70000

80000

MySQL 5.6: NoSQL Benchmarking

Mem-cached APISQL

Client Connections

TPS

Clients and Applications

MySQL ServerMemcached Plug-in

innodb_memcached

local cache(optional)

Handler API InnoDB API

InnoDB Storage Engine

mysqld process

SQL Memcached Protocol

Up to 9x Higher “SET / INSERT” Throughput


MySQL Cluster 7.3: Node.js NoSQL API Native JavaScript access to MySQL Cluster

– End-to-End JavaScript: browser to the app & DB

– Storing and retrieving JavaScript objects directly in MySQL Cluster

– Eliminate SQL transformation Implemented as a module for node.js

– Integrates Cluster API library within the web app Couple high performance, distributed apps,

with high performance distributed database Optionally routes through MySQL Server


MySQL Cluster NoSQL API for Node.js & FKs FKs enforced on all APIs:{ message: 'Error', sqlstate: '23000', ndb_error: null, cause: { message: 'Foreign key constraint violated: No parent row found [255]',

sqlstate: '23000', ndb_error: { message: 'Foreign key constraint violated: No parent row found', code: 255, classification: 'ConstraintViolation', handler_error_code: 151, status: 'PermanentError' }, cause: null } }


Next StepsLearn More• www.mysql.com/cluster• Authentic MySQL Curriculum:

http://oracle.com/education/mysql

Try it Out• dev.mysql.com/downloads/cluster/

Let us know what you think• clusterdb.com• @clusterdb• forums.mysql.com/list.php?25


OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds

Software

jettison relational

parent row

acid transactions

rights reserved

sarahmei https

liker commenter

time copyright

cluster