MySQL Document Store

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |

MySQL as a Document Store Mario Beck MySQL Sales Consulting Manager EMEA [email protected]


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Program Agenda

MySQL Document Store

MySQL Future

1

2

3


Program Agenda


MySQL Future

1

2

4


Today’s Challenges

• Developers want to move faster

• Time to market has a premium value

• Rapid prototyping, iterate fast…

5


InnoDB Storage Engine

mysqld process

Advantages of SQL and NoSQL

NoSQL

Simple access patterns

Compromise on consistency for performance

Ad-hoc data format

Simple operation

SQL

Complex queries with joins

ACID transactions

Well defined schemas

Rich set of tools

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 6


Our Objective: Leverage the Infrastructure


Clients and Applications


mysqld process

Firewall

Audit

Encryption

Authentication

Online Backup

Monitoring

Integration

Support

3rd Party Tools 3rd Party Tools


3rd Party Tools

NoSQL



Ad-hoc data format

Simple operation

SQL


ACID transactions


Rich set of tools


NEW! MySQL Document Store

• Native JSON Documents in MySQL 5.7

– Schema-less Document Storage

• X Protocol (MySQL 5.7.12 DMR)

– Implemented by X Plugin to Extend MySQL Server as a Document Store

• X Dev API – SQL and Document CRUD Operations

– Implemented in Connector/Node.js, Connector/J, Connector/Net

• MySQL Shell

– Javascript, Python, SQL modes


Indexing JSON data

• Use Functional Indexes,

• STORED and VIRTUAL types are supported

9

CREATE TABLE employees (data JSON); ALTER TABLE employees ADD COLUMN name VARCHAR(30) AS (JSON_UNQUOTE(data->”$.name”)) VIRTUAL, ADD INDEX name_idx (name); SELECT data FROM employees WHERE name = ”Simone”; SELECT data FROM employees WHERE JSON_UNQUOTE(data->”$.name”) = ”Simone”;

• Column – generated from the expression – extracts attribute

• Index on virtual column

• Optimizer uses index automatically


Architecture

10

MySQL

Plugins

X Protocol Plugin Memcached Plugin Core

X Protocol Std Protocol

X Protocol 33060

Std Protocol 3306

SQL API CRUD API

X and Std Protocols

MySQL Shell


New! MySQL X DevAPI

• Modern: fluent API, method chaining

• Stateless sessions enable transparent scaling to multi-server environments

• SQL support

• CRUD for Collections of Documents and Tables

– Documents as simple basic domain objects

– Search expressions match SQL SELECT expressions

• Implemented in MySQL Shell & MySQL Connectors – NEW! MySQL Connector/node.js

– MySQL Connector/J

– MySQL Connector/Net


New! MySQL Shell

• Integrated Development and Administration Shell

• Exposes New X DevAPI

• Multi-Language scripting

– JavaScript, Python, and SQL

• Configurable results formats

– Traditional Table, JSON, Tab Separated

12

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 13

tomas@localhost $ mysqlsh --uri root@localhost/test Creating an X Session to root@localhost:33060/test Enter password: Default schema `test` accessible through db. …

Currently in JavaScript mode. Use \sql to switch to SQL mode and execute queries. mysql-js> db.createCollection("posts"); <Collection:posts> mysql-js> db.posts.add({"title":"Hello World", "text":"First post!"}) Query OK, 1 item affected (0.03 sec) mysql-js> db.posts.find("title = 'Hello World'").sort(["title"]); [ { "_id": "8202bda28206e611140b3229389b6526", "text": "First post!", "title": "Hello World" } ] 1 document in set (0.01 sec)


Under the Hood


Collections are tables

Tables with: - JSON column - Generated Column

Create a Collection


Under The Hood II

16

• X-Plugin translates CRUD -> SQL

• No code changes in core parts of MYSQL

Create Document (CRUD)

Look inside general-log


Under The Hood III

17

Read Document (CRUD)

Look inside general-log


CRUD Operations – NoSQL/Document and SQL/Relational

Operation Document Relational

Create Collection.add() Table.insert()

Read Collection.find() Table.select()

Update Collection.modify() Table.update()

Delete Collection.remove() Table.delete()

18


CRUD Operations NoSQL/Document Javascript Java

C# NodeJS


CRUD Operations in php NoSQL/Document

/* open a new connection */ $session = mysql_xdevapi\getSession("localhost","root", "secretpwd"); /* CREATE a new collection and document*/ $schema = $session->getSchema("products"); $collection = $schema->createCollection("best_products"); $product = array('name' => 'superdent', ' quantity' => 42); $collection->add($product)->execute(); /* READ documents */ $largestock = $collection->find()->sort(“quantity desc")->limit(3)->execute();


CRUD Operations in php NoSQL/Document

/* UPDATE a document*/ $collection->modify("name like 'SuperDent'")->set(“quantity", 15)->execute(); $collection->modify("name like :pr_name")->bind(["pr_name" => $productName])-> set(“quantity", 15)->execute(); $collection->modify(“quantity < 10")->set("comment", "order more")->execute(); /* DELETE documents */ $collection->remove("name like 'Super%'")->execute();


CRUD Operations SQL/Relational

22

Javascript Java

C# NodeJS


Program Agenda


MySQL Future

1

2

23


Vision

24


“Be the most popular open source database for scale-out”

25


“Be the most popular open source database for scale-out”

26


3 MySQL will focus on three things.

27


Scale-Out

Ease-of-Use

Out-of-Box Solution

MySQL


Ease-of-Use

• Download, Install, HA & Scale-Out in 15 minutes

• Single Interface for Everything MySQL

• Easy to Setup, Scale-Out, Manage & Monitor

• Excellent Quality

MySQL


Out-of-Box Solution

• Integrated Solution vs. Individual Components

• Designed & Developed Together

• Tested & Delivered Together

• Managed & Monitored Together

MySQL


Scale-Out

• Maintain World-Class Performance

• Rock Solid, Server-Side HA With Auto-Failover

• Read Scale-Out With Replication

• Write Scale-Out With Sharding

MySQL


4 Rollout will happen in four steps.

32


Read Scale-Out

Async Replication + Auto Failover

Write Scale-Out

Sharding

S1

S2

S3

S4

MySQL Vision – 4 Steps

Timeline


Relational & Document Model

MySQL HA

Out-Of-Box HA


Introducing …

34


Scale-Out High Performance

Ease-of-Use Built-in HA

Out-of-Box Solution Everything Integrated

MySQL InnoDB

cluster


A single product: MySQL • All components created together

• Tested together

• Packaged together

Flexible and Modern • C++ 11

• Protocol Buffers

• Developer friendly

MySQL InnoDB Cluster – Goals

Easy to use • A single client: MySQL Shell

• Easy packaging

• Homogenous servers

Scale-out • Sharded clusters

• Federated system of N replica sets

• Each replica set manages a shard


MySQL

InnoDB cluster

MySQL InnoDB Cluster – Architecture - S2

M

M M

MySQL Connector

Application

MySQL Router

MySQL Connector

Application

MySQL Router

MySQL Shell

HA

Group Replication


S1 S2 S3 S4 S…

M

M M

MySQL Connector

Application

MySQL Router

MySQL Connector

Application

MySQL Router

MySQL Shell

HA

MySQL InnoDB Cluster – Architecture - S3 MySQL

InnoDB cluster

Read-Only Slaves


S1 S2 S3 S4 S…

M

M M

MySQL Connector

Application

MySQL Router

MySQL Connector

Application

MySQL Router

MySQL Shell

HA

Rep

licaS

et (

Shar

d 1

)

S1 S2 S3 S4 S…

M

M M

MySQL Connector

Application

MySQL Router

HA

Rep

licaS

et (

Shar

d 2

)

S1 S2 S3 S4

M

M M

HA

Rep

licaS

et (

Shar

d 3

)

MySQL Connector

Application

MySQL Router

MySQL InnoDB Cluster – Architecture - S4 MySQL

InnoDB cluster

…


MySQL Shell – Deploy MySQL Instances

shell> mysqlsh

mysql-js> dba.deployLocalInstance(3306)

mysql-js> dba.deployInstance(‘hanode2:3306’)

mysql-js> dba.deployInstance(‘hanode3:3306’)

40


MySQL Shell – Create an InnoDB Cluster

mysql-js> \connect root@hanode1:3306

mysql-js> cluster = dba.createCluster(‘NewAppCluster')

mysql-js> cluster.addInstance('root@hanode2:3306')

mysql-js> cluster.addInstance('root@hanode3:3306')

41


MySQL Shell – Add a MySQL Router

shell> mysqlrouter --bootstrap hanode1:3306

shell> mysqlrouter &

shell> mysqlsh --uri root@localhost:6446

42



mysqld process

Advantages of SQL and NoSQL

NoSQL



Ad-hoc data format

Simple operation

SQL


ACID transactions


Rich set of tools



Our Objective: Leverage the Infrastructure




mysqld process

Firewall

Audit

Encryption

Authentication

Online Backup

Monitoring

Integration

Support



3rd Party Tools

NoSQL



Ad-hoc data format

Simple operation

SQL


ACID transactions


Rich set of tools


Our Objective: Empower Application Developers




mysqld process

Firewall

Audit

Encryption

Authentication

Online Backup

Monitoring

Integration

Support



3rd Party Tools

NoSQL



Ad-hoc data format

Simple operation

SQL


ACID transactions


Rich set of tools


Resources

Topic Link(s)

MySQL as a Document Database http://dev.mysql.com/doc/refman/5.7/en/document-database.html

MySQL Shell http://dev.mysql.com/doc/refman/5.7/en/mysql-shell.html http://dev.mysql.com/doc/refman/5.7/en/mysqlx-shell-tutorial-javascript.html http://dev.mysql.com/doc/refman/5.7/en/mysqlx-shell-tutorial-python.html

X Dev API http://dev.mysql.com/doc/x-devapi-userguide/en/

X Plugin http://dev.mysql.com/doc/refman/5.7/en/x-plugin.html

MySQL JSON http://mysqlserverteam.com/tag/json/ https://dev.mysql.com/doc/refman/5.7/en/json.html https://dev.mysql.com/doc/refman/5.7/en/json-functions.html

46

http://mysqlserverteam.com/




http://dev.mysql.com/doc/refman/5.7/en/mysqlx-shell-tutorial-javascript.html











http://dev.mysql.com/doc/refman/5.7/en/mysqlx-shell-tutorial-python.html







http://dev.mysql.com/doc/x-devapi-userguide/en/





http://dev.mysql.com/doc/refman/5.7/en/x-plugin.html



http://mysqlserverteam.com/tag/json/

https://dev.mysql.com/doc/refman/5.7/en/json.html

https://dev.mysql.com/doc/refman/5.7/en/json-functions.html




Thank you!

MySQL Document Store

Technology