Introducing the MySQL Document Store

Copyright©2014Oracleand/oritsaffiliates.Allrightsreserved.|

IntroducingtheMySQLDocumentStore

AlfredoKojima–Sr.Dev.ManagerMikeFrank–ProductManagementDirectorOracle

OracleConfidenNal–Internal/Restricted/HighlyRestricted


SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirecNon.ItisintendedforinformaNonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncNonality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andNmingofanyfeaturesorfuncNonalitydescribedforOracle’sproductsremainsatthesolediscreNonofOracle.

2


Today’sAgenda

RelaNonalDatabases,DocumentDatabasesandMySQL

MySQLJSONSupport

DocumentUseCases

TheXDevAPI

Ge\ngitallworkingtogether

1

2

3

4

3

5


RelaNonalDatabases• DataIntegrity– NormalizaNon– Constraints(foreignkeysetc)

• Atomicity,Consistency,IsolaNon,Durability-ACID– TransacNons

•  SQL– Powerful,OpNmizableQueryLanguage– DeclarewhatyouwantandtheDBwillfindoutthemostefficientwaytogetittoyou

OracleConfidenNal–Restricted 4


Plus…• MySQLhasbeenaroundsince1995• Ubiquitous• Pregymuchastandard•  Scalable• Whenthereareissues,theyareknownandunderstood•  Largebodyofknowledge,fromsmalltoBIGdeployments



DocumentDatabases


•  Schemaless– noschemadesign,normalizaNon,foreignkeys,constraints,datatypesetc– fasteriniNaldevelopment

•  Flexibledatastructures– nestedarraysandobjects– somedataissimplynaturallyunstructuredorcannotbemodeledefficientlyintherelaNonalmodel(hierarchies,productDBetc)– persistobjectswithoutORMs


DocumentDatabases(Cont.)


•  JSON– Closertothefrontend– "naNve"inJavaScript– Node.jsandfullstackJavaScript

•  Easytolearn,easytouse


RelaNonalvsDocumentDatabases

Whynotboth?




MySQLJSONSupport

DocumentUseCases

TheXDevAPI


9

1

2

3

4

5


MySQL5.7:JSONSupport• NaNveJSONdatatype•  StoreJSONvalues(objects,arraysandsimplevalues)inMySQLtables• BinaryJSONstorageformat• Conversionfrom"naNve"SQLtypestoandfromJSONvalues•  JSONManipulaNonfuncNons– Extractcontents(JSON_EXTRACT,JSON_KEYSetc)– Inspectcontents(JSON_CONTAINSetc)– Modifycontents(JSON_SET,JSON_INSERT,JSON_REMOVEetc)– Createarraysandobjects(JSON_ARRAY,JSON_OBJECT)– Searchobjects(JSON_SEARCH)



MySQL5.7:JSONSupport(cont.)•  InlineSQLJSONpathexpressionsSELECT doc->'$.object.array[0].item' FROM some_table

• Booleanoperators(compareJSONvaluesetc)– foo=doc->'$.field'



MySQL5.7:JSONSupport(cont.)• Plus…• Generated/VirtualColumns– IndexJSONdata– ForeignkeystoJSONdata– SQLviewsforJSONdata




MySQLJSONSupport

DocumentUseCases

TheXDevAPI


13

1

2

3

4

5


ExtracNngJSONfromaRelaNonalDBRela+onalIn,Rela+onal+DocumentOut

• DatastoredinrelaNonaltables,butfrontendusesJSON•  JSONdirectlymapstonaNvedatastructuresinmanylanguages– OveneasierforapplicaNoncodetouse– JavaScript,Python,Rubyetc– InbrowserJavaScript



ExtracNngJSONfromaRelaNonalDBRela+onalIn,Rela+onal+DocumentOut

•  SQLFuncNonstoconstructJSON– JSON_OBJECT(),JSON_ARRAY()

•  Ex.:SELECT JSON_OBJECT('cust_id', id, 'name', name, 'email', email) FROM customer;

CREATE VIEW customer_json AS

SELECT JSON_OBJECT('cust_id', id, 'name', name, 'email', email) as doc FROM customer;

SELECT * FROM customer_json;

• UpdatesandinsertssNllhappenthroughthetablecolumns



UsingMySQLasaJSONDocumentContainerDocumentIn,Rela+onal+DocumentOut

• VirtuallySchemaless– Unstructureddata– Noclear,fixedstructureforthedata…recordscanhavedifferentfields– Ovendatathatisnotinvolvedinbusinessrules– Examples:"product_info","properNes","opNons"etc

• DatadoesnotmapcleanlyintoarelaNonalmodel(arrays,hierarchicaldataetc)• Prototyping



MySQLasaJSONDocumentContainer


Example:"properNes"table

hgps://www.mediawiki.org/wiki/Manual:Database_layout


MySQLasaJSONDocumentContainer


Example:"product_info"tableproduct_id a<ribute value

9 size M

9 color red

9 fabric cogon

11 flavour strawberry

12 capacity 128GB

12 speedclass class10

13 connecNvity Wi-Fi

13 storage 64GB

13 screensize 8.9"

13 resoluNon 2560x1600(339ppi)

13 bagerylife 12hours

{"product_id":9,"size":"M","color":"red","fabric":"cogon"},{"product_id":11,"flavour":"strawberry"},{"product_id":12,"capacity":"128GB","speedclass":"class10"},{


MySQLasaJSONDocumentContainerDocumentIn,Rela+onal+DocumentOut

• AnordinaryMySQLtablewithasingleJSONdatacolumn• GeneratedcolumnsallowSQLenginetolookinsidetheJSONdata– Virtualcolumns– PrimaryKeys– Indexes– ForeignKeys

• WritesontheJSONcolumn• ReadsprimarilyfromtheJSONcolumns



HybridRelaNonalandJSONRela+onal+DocumentIn,Rela+onal+DocumentOut

• DatabaseismostlyrelaNonal•  SomepartsofthedatabaseareunstructuredordoesnotmodelcleanlyasrelaNonal•  JSONcolumnsinrelaNonaltables• QueriescanmixandmatchJSONandcolumndata•  EvoluNonpathforDocumentbasedapplicaNons



HybridRelaNonalandJSONRela+onal+DocumentIn,Rela+onal+DocumentOut

product (product_id, name, category_id, …);

product_info (product_id, attribute, value);


product (product_id, name, category_id, …, attributes JSON);



MySQLJSONSupport

DocumentUseCases

TheXDevAPI


1

2

3

4

22

5


DocumentOperaNonsviaSQL• Powerful• Allowscomplexqueries• But…sNlldifficulttouse



DocumentOperaNonsviaSQL

CREATE TABLE product (

id VARCHAR(32) GENERATED ALWAYS AS (JSON_EXTRACT(doc, '$.id')) STORED,

doc JSON

);

INSERT INTO product VALUES (1, '{…}');

SELECT * FROM product WHERE JSON_EXTRACT(doc, '$.field') = value;

etc.



TheXDevAPI• AbstracNonoverSQL•  Focusedon4basicCRUDoperaNons(Create,Read,Update,Delete)•  Fluent,NaNveLanguageAPI• NoknowledgeofSQLneeded• XProtocol– CRUDrequestsencodedatprotocollevel– Requestdetails"visible"(vs"opaque"SQLstrings)



CollecNonandSchemaOperaNons• GetahandletoaSchemamydb = session.getSchema("mydb");

• CreateaCollecNonmydb.createCollection("products");

• Geta(local)referencetoaCollecNonproducts = mydb.getCollection("products");



AddDocument

products.add({"name":"bananas", "color":"yellow"}).execute();



FindDocuments


products.find("color = 'yellow'").sort(["name"]).execute();


ModifyDocuments


products.modify("product_id = 123").set("color", "red").execute();


RemoveDocuments


products.remove("product_id = 123").execute();


XDevAPISessions• XSession– Stateless– CRUDonly,noSQL– AbstractstheconnecNon

• NodeSession– DirectconnecNontoadatabasenode– AllowsCRUDandSQL



OtherOperaNonsonCollecNons• CreateanIndexdb.post.createIndex("email").field("author.email","text(30)",false)



CRUDOperaNons–NoSQL/DocumentandSQL/RelaNonal

Opera+on Document Rela+onalCreate CollecNon.add() Table.insert()Read CollecNon.find() Table.select()Update CollecNon.modify() Table.update()Delete CollecNon.remove() Table.delete()

33



MySQLJSONSupport

DocumentUseCases

TheXDevAPI


1

2

3

4

34

5


5.7.12DevelopmentPreviewRelease• MySQL5.7.12withDocumentStoreplugin• MySQLShell1.0.3• Connector/J7.0• Connector/Net7.0• Connector/Node.js1.0



MySQL5.7,Connectors,Drivers,andProtocols

36

MySQL

Plugins

XProtocolPlugin MemcachedPluginCore

MySQLConnectorsandDrivers

XProtocolStdProtocol

Memcacheddriver

XProtocol33060

StdProtocol3306

SQLAPI CRUDandSQLAPIs

MemcacheProtocol

XandStdProtocols

MySQLShell


XDevAPIConnectors–MySQLConnector/Java7.0uri = "mysql:x://localhost:33060/test?user=user&password=mypwd"'; !

XSession session = new MysqlxSessionFactory().getSession(uri); !

Schema schema = session.getDefaultSchema();!

// document walkthrough!

Collection coll = schema.createCollection("myBooks", true); !

DbDoc newDoc = new DbDoc().add("isbn", !new JsonString().setValue("12345")); !

newDoc.add("title", !new JsonString().setValue("Effi Briest")); !

newDoc.add("author", !new JsonString().setValue("Theodor Fontane")); !

newDoc.add("currentlyReadingPage", !new JsonNumber().setValue(String.valueOf(42))); !

coll.add(newDoc).execute(); !

DocResult docs = coll.find("title = 'Effi Briest' and currentlyReadingPage > 10").execute(); !

DbDoc book = docs.fetchOne(); !

System.err.println("Currently reading "! + ((JsonString)book.get("title")).getString() ! + " on page " ! + ((JsonNumber)book.get("currentlyReadingPage")).getInteger()); !

!

// increment the page number and fetch it again!

coll.modify("isbn = 12345"). ! set("currentlyReadingPage", ! ((JsonNumber)book.get("currentlyReadingPage")).getInteger() + 1).execute(); !

docs = coll.find("title = 'Effi Briest' and currentlyReadingPage > 10").execute(); !

book = docs.fetchOne(); !

System.err.println("Currently reading " ! + ((JsonString)book.get("title")).getString() ! + " on page "! + ((JsonNumber)book.get("currentlyReadingPage")).getInteger()); !



XDevAPIConnectors–MySQLConnector/Net7.0using(XSessionsession=

MySQLX.GetSession("mysqlx://test:test@localhost:33060")){stringschemaName="test";SchematestSchema=session.GetSchema(schemaName);if(testSchema.ExistsInDatabase())session.DropSchema(schemaName);session.CreateSchema(schemaName);//insertsomedocsCollectioncoll=testSchema.CreateCollection("myDocs");vardocs=new[]{new{_id=1,title="Book1",pages=20},new{_id=2,title="Book2",pages=30},new{_id=3,title="Book3",pages=40},new{_id=4,title="Book4",pages=50},};

Resultr=coll.Add(docs).Execute();Console.WriteLine("Docsadded:"+r.RecordsAffected);//modifysomevaluesr=coll.Modify("_id=:ID").

Bind("Id",2).Set("pages","25").Execute();Console.WriteLine("Docsmodified:"+r.RecordsAffected);//removeabookr=coll.Remove("_id=:ID").Bind("Id",4).Execute();Console.WriteLine("Docsremoved:"+r.RecordsAffected);//listtheresultsvarresult30orMore=coll.Find("pages>20").

OrderBy("pagesDESC").Execute().FetchAll();foreach(vardocinresult30orMore){Console.WriteLine(doc.ToString());}}



XDevAPIConnectors–MySQLConnector/Node.js1.0NEW!const mysqlx = require('mysqlx'); !!mysqlx.getSession({ ! host: 'localhost', ! dbUser: 'myuser', ! dbPassword: 'secret' !}).then(session => { ! const collection = session.getSchema('myschema').getCollection('mycollection'); ! return Promise.all([ ! collection.add({ foo: "bar", something: { nested: [1,2,3,4] } }).execute(); ! session.close(); ! ]) !}).catch(err => { ! console.log(err); !}); !!

!const collection = session.getSchema('myschema').getCollection('questions'); !collection.find("answer == 42") ! .orderBy("foo DESC") ! .limit(10) ! .execute(doc => console.log(doc)) // print the document received, callback called for each doc! .then(() => console.log("All done") // Promise resolves when all data is there! .catch((err) => console.log("Oups, an error", err); ! !



MySQLShell



MySQLPluginforVisualStudio



ResourcesTopic Link(s)

MySQLasaDocumentDatabase hgp://dev.mysql.com/doc/refman/5.7/en/document-database.html

MySQLShell hgp://dev.mysql.com/doc/refman/5.7/en/mysql-shell.htmlhgp://dev.mysql.com/doc/refman/5.7/en/mysqlx-shell-tutorial-javascript.htmlhgp://dev.mysql.com/doc/refman/5.7/en/mysqlx-shell-tutorial-python.html

XDevAPI hgp://dev.mysql.com/doc/x-devapi-userguide/en/

XPlugin hgp://dev.mysql.com/doc/refman/5.7/en/x-plugin.html

MySQLJSON hgp://mysqlserverteam.com/tag/json/hgps://dev.mysql.com/doc/refman/5.7/en/json.htmlhgps://dev.mysql.com/doc/refman/5.7/en/json-funcNons.html

Blogs hgp://mysqlserverteam.com/category/docstore/

42


ThankYou!

[email protected]