MySQL Day Paris 2016 - MySQL as a Document Store

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |

MySQL as a Document Store

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Olivier DasiniMySQL Principal Solutions Architect [email protected]@freshdaz

mailto:[email protected]


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

2


Today’s Challenges• Developers want to move faster• Time to market has a premium value• Rapid prototyping, iterate fast…

3


Today’s Challenges (cont.)• Relational databases ask for schema up front– Potentially saving you time in the future

– Less variations; less code to handle edge cases

–Added cost with each schema change

• NoSQL databases do not ask for schema– Saving you time up front

–But potentially adding operational costs over time

–No cost per schema change

4


Today’s Challenges (cont.)• The Most popular NoSQL database uses similar data structures to

relational databases (B+TREE)– Data structure influences key performance characteristics not data model

– No inherent scalability advantages

– Yet behind on key functionality

• Standing up multiple technologies adds complexity for operations teams

5


Why can you not…• Have both schema-less and schema in the same technology stack?

• One that checks all the boxes of all stakeholders:

6

Developers:[ x ] Schemaless[ x ] Rapid Prototyping/Simpler APIs[ x ] Document Model

Operations:[ x ] Performance Management/Visibility[ x ] Robust Replication, Backup, Restore[ x ] Comprehensive Tooling Ecosystem

Business Owner:[ x ] Don’t lose my data = ACID transactions[ x ] Capture all my data = Extensible/Schemaless[ x ] Products On Schedule/Time to Market = Rapid Development


We <3 Schemaless• MySQL as a Document Store• All the existing features of MySQL–Replication

– InnoDB

– Performance Schema

• With the addition of schemaless –Documents using JSON

– Easy to program CRUD APIs

7


MySQL – a One Stop Shop

• Expensive to manage many data stores

• Better few databases – more flexibility– More developers • Can harness both NoSQL and SQL savvy

– More DBAs• No shortage of highly experienced MySQL DBAs

– Less training – don’t need to learn many products

– Cross data store exchange – easier to move from from docs to tables etc.

– One connector/driver needed for apps

Combining Relational and Document Stores

8


• One Extensible database– Do more with the MySQL database

– Many can manage (with deep skills)

– Stable

– Cost-effective

– Easy to move data between like database types

– Fewer Drivers

– Few Tools

– SQL works, CRUD works

– Operational and Analytical Together

• Many different databases– Requires larger skill repertoire, more

complex development …• Harder to find deep skills

– Many Drivers

– Many Tools

– More effort to share and exchange data

– It’s a lot more work

– Operational and Analytical Separate

One Database Many Models VS Many Databases Many Models

9


NEW! MySQL as a Document Store• Native JSON Documents in MySQL 5.7– JSON data type

– Schema-less Document Storage

– Indexes on JSON documents (w/ Generated Columns)

• X Protocol (MySQL 5.7.12+)– Implemented by X Plugin to Extend MySQL Server as a Document Store

• X Dev API– SQL and NoSQL CRUD Operations

– Implemented in Connectors : Node.js, Java, .Net, Python, C/C++

– Supports Combined Document and Relational

• MySQL Shell– JavaScript (\js), Python (\py), SQL (\sql) modes


MySQL 5.7, Connectors, Drivers, and Protocols

11

MySQLMySQL

Plugins

X Protocol Plugin Memcached PluginCore

MySQL Connectors and Drivers

X ProtocolStd Protocol

Memcached driver

X Protocol33060

Std Protocol3306

SQL API CRUD and SQL APIs

MemcachedProtocol

X and StdProtocols

MySQLShell


MySQL X Protocol

• Asynchronous API support – Supporting Parallelism and Batching

– Pipelining (streaming) - Send multiple requests, Reduce Round Trips

– Push Notifications - Beyond the Request/Response model

• Middleware Friendly

– Routing, Sharding, Read Write Splitting

• Uses open standards: TLS, SASL, Protobuf etc

– Proven, community friendly

• Supports both CRUD and SQL operations

12

http://mysqlserverteam.com/mysql-5-7-12-part-2-improving-the-mysql-protocol/

http://mysqlserverteam.com/mysql-5-7-12-part-2-improving-the-mysql-protocol/


MySQL Shell – Key Features

• Multi-Language - JavaScript, Python, and SQL languages

• Supports both Document and Relational Models

• Interactive and Batch processing modes

• Three results formats – Traditional Table, JSON, Tab Separated,

• Exposes Full Development API– Sessions

– Schemas

– Collections

– Tables

Integrated Development and Administration Shell

13

– CRUD on Tables and Collections– SQL execution– Result Processing– Parameter Binding


MySQL Shell – Key Features (cont.)

• Stored Sessions – for 1 to Many servers

• Supports Multiple Connections per Session

• Support all 3 New Session types– Xsession – CRUD only session (Connected Server is “Abstacted”)

– Node – SQL and CRUD (Connected Server is “Fixed”)

– Standard MySQL session – SQL (Connected Server is “Fixed”)

• Optional Session Logging – – Set Logging Level, Verify state, Review history

Global Sessions

14


MySQL X DevAPI

• Modern: fluent API, method chaining

• Stateless sessions enable transparent scaling to multi-server environments

• SQL support

• CRUD for Collections of Documents and Tables

– Documents as simple basic domain objects

– Search expressions match SQL SELECT expressions

• Implemented in MySQL Shell & MySQL Connectors

– NEW! MySQL Connector/node.js

– MySQL Connector/J | MySQL Connector/C++

– MySQL Connector/Net | MySQL Connector/Python

15

http://dev.mysql.com/doc/x-devapi-userguide/en/

http://dev.mysql.com/doc/x-devapi-userguide/en/


Operation Document Relational

Create Collection.add() Table.insert()

Read Collection.find() Table.select()

Update Collection.modify() Table.update()

Delete Collection.remove() Table.delete()

16

• Use SQL, CRUD APIs – Document (NoSQL) and Relational (SQL), or “All of the Above”

– All of this is in addition to the Classic APIs

MySQL Connectors include X DevAPI


MySQL Documents and Collections• Collections are containers for documents – These documents share a purpose

– Possibly share one or more indexes

– Each collection has a unique name

– Exists within a single schema

• Within a Collection you can –Add(), Find(), Modify(), and Remove() - JSON documents

• Collections can be–Create(), List(), Drop()

21


Collection Search – find(), bind(), fields()

• Supports many operators to specify searches

– ||, &&, XOR, IS, NOT, BETWEEN, IN, LIKE, !=, <>, >, >=, <, <=, &, |, <<, >>, +, -, *, /, ~, %.

• Searching

– db.CountryInfo.find("GNP > 500000 and demographics.Population < 100000000")

– db.CountryInfo.find("GNP*1000000/demographics.Population > 30000")

• Binding - bind()

– db.CountryInfo.find("Name = :country").bind("country", "Italy")

• Project Results – fields() – returns specific fields

– db.CountryInfo.find("Name = :country").bind("country", "Italy")

22


Installation needs

1) MySQL 5.17.12 or later

2) Install X plugin mysql> INSTALL PLUGIN mysqlx SONAME “mysqlx.so”;

3) Install mysqlsh

24

https://dev.mysql.com/doc/refman/5.7/en/document-store-setting-up.html


CRUD Operations NoSQL/Document

Javascript Java

C#NodeJS


CRUD OperationsSQL/Relational

26

Javascript Java

C#NodeJS

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 27

Let's play with MySQL Shell...


$ mysqlsh uri root@localhost/docCreating an X Session to root@localhost:33060/docEnter password: Default schema `doc` accessible through db.…Currently in JavaScript mode. Use \sql to switch to SQL mode and execute queries.mysqljs> session<XSession:root@localhost:33060/doc>mysqljs> db<Schema:doc>

mysqljs> db.getCollections(){

}mysqljs> db.createCollection("posts");<Collection:posts>mysqljs> db.getCollections(){ "posts": <Collection:posts>}mysqljs> db.posts.add({"title":"MySQL rocks", "text":"My first post!"})Query OK, 1 item affected (0.02 sec)


mysqljs> db.posts.add({"title":"Polyglot database", "text":"With MySQL we can handle SQL and NoSQL data"})Query OK, 1 item affected (0.02 sec)mysqljs> db.posts.find();[ { "_id": "72bfbd1958a7e6119733dc53609b8b8d", "text": "My first post!", "title": "MySQL rocks" }, { "_id": "b2c4380859a7e6119733dc53609b8b8d", "text": "With MySQL we can handle SQL and NoSQL data", "title": "Polyglot database" }]2 documents in set (0.01 sec)mysqljs> db.posts.find("title = 'Polyglot database'");[ { "_id": "b2c4380859a7e6119733dc53609b8b8d", "text": "With MySQL we can handle SQL and NoSQL data", "title": "Polyglot database" }]1 document in set (0.00 sec)

https://dev.mysql.com/doc/refman/5.7/en/document-store-setting-up.html


An SQL view gives...


switch to a node session to be able to use sql commandsmysqljs> \connect_node root@localhost/docCreating a Node Session to root@localhost:33060/doc

mysqljs> \sqlSwitching to SQL mode... Commands end with ;mysqlsql> SHOW TABLES;++| Tables_in_doc |++| posts |++

mysqlsql> SHOW CREATE TABLE posts;…CREATE TABLE posts ( doc json DEFAULT NULL, _id varchar(32) GENERATED ALWAYS AS (json_unquote(json_extract(`doc`,'$._id'))) STORED NOT NULL, PRIMARY KEY (_id)) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4

mysqlsql> SELECT doc FROM posts;++| doc |++| {"_id": "72bfbd1958a7e6119733dc53609b8b8d", "text": "My first post!", "title": "MySQL rocks"} || {"_id": "b2c4380859a7e6119733dc53609b8b8d", "text": "With MySQL we can handle SQL and NoSQL data", "title": "Polyglot database"} |++


MySQL Document Store

✔ Built on Proven SQL/InnoDB/Replication

✔ Schema-less/Relational/Hybrid

✔ ACID/Transactions

✔ CRUD/JSON/Documents

✔ Modern Dev API

✔ Modern/Efficient Protocol

✔ SQL Queries/Analytics over JSON Documents

✔ Transparent and Easy HA/Scaling/Sharding

35


ResourcesTopic Link(s)

MySQL as a Document Storehttp://dev.mysql.com/doc/refman/5.7/en/document-store.htmlhttp://mysqlserverteam.com/category/docstore/

MySQL Shellhttp://dev.mysql.com/doc/refman/5.7/en/mysql-shell.htmlhttp://dev.mysql.com/doc/refman/5.7/en/mysqlx-shell-tutorial-javascript.htmlhttp://dev.mysql.com/doc/refman/5.7/en/mysqlx-shell-tutorial-python.html

X Dev API http://dev.mysql.com/doc/x-devapi-userguide/en/

X Plugin http://dev.mysql.com/doc/refman/5.7/en/x-plugin.html

MySQL JSONhttps://dev.mysql.com/doc/refman/5.7/en/json.htmlhttps://dev.mysql.com/doc/refman/5.7/en/json-functions.htmlhttp://mysqlserverteam.com/tag/json/

36


Thank you!

MySQL Day Paris 2016 - MySQL as a Document Store

Technology