Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
MariaDB versus NoSQL; Best of 2 worlds ?
MariaDB or NoSQL(or both)
Oliopäivät 2012Michael “Monty” WideniusEntrepreneur, MariaDB [email protected]://mariadb.com/
Introducing Maria
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
What is MariaDB
● Open Source, binary compatible, superset of MySQL:● True drop in replacement of MySQL
● Same data on disk and on the wire.● Same file names, same sockets and ports.
● Created and maintained by the same people that created MySQL.● Open development: Developed together with the community.● True fork, not just a patch set upon MySQL.
● MariaDB is not depending on MySQL for future development.● More plugins, more features, faster, better code quality.
● 30 man years more development than MySQL ● GPL-only server license (no closed source extensions)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Why MariaDB was created
“Save the People, Save the Product”
● To keep the MySQL talent together● To ensure that a free version of MySQL always exists● To get one community developed and maintained branch● Work with other MySQL forks/branches to share knowhow
and code
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
MariaDB server releases
● MariaDB 5.1 was released as stable in February 2010● MariaDB 5.2 was released as stable in November 2010● MariaDB 5.3 was released as stable in April 2012● MariaDB 5.5 was released as stable in April 2012● MariaDB 10.0 was released as alpha in November 2012
● MariaDB-Galera (multi-master) was released as Alpha in September 2012 after a lot of testing.
●Gamma/stable to be released shortly (December)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
MyISAM Segmented key cache
● Blue line is without segmented key cache.● Solves one of the major read bottlenecks for MyISAM● We see up to 250% performance gain depending on the
amount of concurrent users.● Fix applies to all MyISAM usage with many readers!
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
query_size=1 query_size=2 query_size=30
500
1000
1500
2000
2500
3000
quer
y tim
e, s
ec (
smal
ler
is b
ette
r)
New Batched Key Access Speedups
● Join benchmark with BKA
select max(l_extendedprice) from orders, lineitem whereo_orderdate between $DATE1 and $DATE2 andl_orderkey=o_orderkey
10x7x
3x
BKA, join_buffer_size=32M
Regular join
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Group commit, verified
Source: Marc Callaghan's facebook blog for a server with 400 microsecond fsync latency
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
New thread pool for 5.5
8 16 32 64 128 256 512 1024 2048 40960
1000
2000
3000
4000
5000
6000
7000
8000
pool-of-threadsthread-per-connection
Concurrent Clients
OL
TP
(ro
) T
hro
ug
hp
ut,
tp
s
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
MariaDB and TokuDB
MariaDB and Tokutek have agreed to make TokuDB a native plugin in MariaDB 5.5 and MariaDB 10.0 by end of Q1 2013.
This means that the official MariaDB binary will be able to dynamically load the TokuDB storage engine directly (no patches needed for MariaDB).
TokuDB will be added to the MariaDB buildbot test suite to ensure that the combination is properly tested on all supported platforms.
TokuDB will be available for download from https://downloads.mariadb.org/ together with MariaDB.
About TokuDB
- TokuDB uses Fractal Tree® indexing to improve insert and query speed, compression, replication performance, and online schema flexibility.
- TokuDB is created by Tokutek Inc. See www.tokutek.com for details.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Connect storage engine
MariaDB 10.0 will include the Connect storage engine by Olivier Betrand.
With the connect storage engine you can read, write and update files in a lot of different storage formats:● Various fixed and dynamic text formats● .DBF (dBASE format)● .CSV● .INI● .XML● ODBC ; Table extracted from an application accessable
with ODBC
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Why MariaDB instead of MySQL
● MariaDB has 30 man years of more development than MySQL (and the gap will continue growing).
● MariaDB is maintained by the people that originally created MySQL and has the best knowledge of the MySQL code.
● MariaDB is binary compatible with MySQL, so its trivial to replace MySQL with MariaDB (minutes).
● Reasons to switch to MariaDB● Faster queries thanks to XtraDB (InnoDB plugin fork from Percona), a better optimizer and replication and better code.
● Open source development: Anyone can be part of the development at all stages. Dev meetings are public.
● More features, including critical ones like microseconds, multi-source and dynamic column support.
● Less risk as MariaDB will not remove features like MySQL is doing (thread pool, storage engines, safemalloc (developer feature), etc)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Big data is here
Source: IDC Digital Universe study 2011
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Big data at end of 2011
● 3146M emails (71% spam)● 555M web sites (300M new)● 220M registered domain names● 800M Facebook users (200M new)● 225M twitter accounts (100M active)● 91250M tweets (250M/day, stored in MySQL)● 2100M internet users (591M with broadband)● 1200M active mobile broadband users● 1 trillion video playbacks from youtube● 6 billion photos on Flickr
Data from royal.pingdom.com
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Big data driven by Open Source
● MySQL / MariaDB● SQL database with flexible replication
● Apache Cassandra● Tunable consistency● Keys map to multiple values, which are grouped into column families● CQL language
● MongoDB● Stores structured data as JSON like documents
● CouchDB● MVCC, ACID, eventually consistent
● Hbase (+ Haddop + Hadoop distributed file system)● Written in Java. Compression, in-memory operation and Bloom filters (Is data part
of a set).● Redis
● In memory database (+ snapshots to disk)● Optional durability
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Database usage 2011
MySQL SQL Server PostgreSQL Oracle MongoDB MariaDB DB2 Cassandra Reddis0
10
20
30
40
50
60
70
80
%
Based on 451 Research surveys: Sample 205 DB users
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Predicted MySQL usage
2009 2011 2012 2014 2014 20170
10
20
30
40
50
60
70
80
90
MySQL forks
Survey 2012
Survey 2009
%
Based on 451 Research surveys
Sample 285/205 users
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Big data driven by Open Source
Predicted database usage growth 2012-2017 of total market:● MariaDB (3.9 %)● Apache/Cassandra (3.4 %)● Apache CouchDB (3.0 %)● Apache Hbase (2.9 %)● Redis (2.5 %)● PostgreSQL (2.4 %)● DB (-1.5%)● Oracle (-5.9%)● SQL Server (-9.3%)● MySQL (-26.4%)
Based on 451 Research survey 2012
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Reasons to use NoSQL
The main reasons for using NoSQL are:●Handling of unstructured data (not everything is table and fixed number of columns)
●Faster replication (usually with 'unconventional' shortcuts)
●Fast and easy key/value access●Data is often stored in memory
● The same way MySQL with it's storage engine interface can handle both transactional and datawarehousing, we are extending MariaDB to be a bridge between SQL and NoSQL.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
SQL is catching up
Replication is getting better and faster● MariaDB has multi-master replication (galera) and Tungsten replication is getting traction
● PostgreSQL now includes replication● MariaDB added to 5.3 (one year ago) dynamic columns which
allows each row to have different set of columns.● Twitter, the company that started the NoSQL hype with their
blog about Cassandra, is still using MySQL to store tweets.● Teradata just announced a Hbase connection● The main benefit of most NoSQL solutions is that data is
mostly in memory.● With similar memory resources you can usually keep SQL data in memory too.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
MariaDB as a bridge between SQL and NoSQL
● Up to 50 % faster HANDLER commands (HANDLER READ …)● Up to 530,000 queres/second measured(*)
● HandlerSocket compiled in (Direct access to InnoDB)● Dynamic columns (each row can have different set of columns)
● Building block for NoSQL storage engines● Storage engine for Cassandra
● You can read, write, update and join with Cassandra● We are working on a storage engine for LevelDB
(*) Stephane Varoqui's blog:
http://varokism.blogspot.com/2011/01/20-to-50-improvement-in-mariadb-53.html
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns in MariaDB 5.3
● With dynamic columns all extra columns are stored in one or many packed blobs, maintained by the database.
● You can instantly add more columns, remove or query them for a row.
● You can access columns in the server or retrieve the full blob to the client and manipulate it there.
● You can use virtual columns to create indexes on some values.
●True indexes for dynamic columns is planned for later.● Implemented trough functions to enable use by ODBC etc.● First implementation uses an integer to access columns.
●In MariaDB 10.0 columns can be accessed by name.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns in MariaDB 5.3
● Simple set of functions (available in server and client):●COLUMN_CREATE(column_nr, value,[column_nr,....])●COLUMN_ADD(blob,column_nr, value, [column_nr,...])●COLUMN_DELETE(blob, column_nr, column_nr...);●COLUMN_EXISTS(blob, column_nr);●COLUMN_LIST(blob, column_nr);●COLUMN_GET(blob, column_nr, type);
As a proof of concept we have in MariaDB 10.0 created a storage engine for Cassandra where we use dynamic columns as a bridge.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Reason to not use NoSQL
● Can't (easily) join/combine data● No standardized language; Hard to move data and
applications between systems● Complete lock in to one system● Normally few connections to computer languages or existing
applications that needs a database.● Allows one to initially ignore solving the database layout
problem as one can store data 'as such'● Initially easier to use, MUCH harder later
● No sustainable scalable business models for most open source NoSQL solutions (as most of them relay on the BSD or Apache licenses)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
NoSQL and the money
MySQL, NoSQL and NewSQL revenue
NoSQL$20m
NewSQL$12M
MySQL$171m
NoSQL$215m
CAGR: 82%
NewSQL$112M
CAGR: 75%
MySQL$664m
CAGR: 40%
451 Research
2011 2015 (estimated)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Problems to overcome with BIG data
Most companies don't need 'big data' solutions● Storage (but not normally the big problem)
● Memory (Very expensive)● SSD (Expensive but slower)● Hard disk (Inexpensive. Slow when doing random reads, fast on sequential reads)
● Access patterns● Get it up when things fails
●Recovery; Can take days or weeks●Get the caches warm (you need > 90 % hit rate with buffers!)
● Replication and hot standby●Must have, but makes big data more expensive
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Why big data solutions is a hype
Most companies don't need 'big data' solutions● Most companies will not have as much data as Facebook, Twitter and Walmart.
● Solutions that they have to use are not applicable for others.● In the near future one can run what most companies think is
big data on a few machines:● Memory now costs about $10/G; Anyone can afford 24G-100G RAM.● 512G SSD is now $400 (Read: 500MB/sec 80K IOPS, Write: 260-400M/sec write, 36K IOPS)
● 1.2T fusionIO cards (can be linked to 2.4T) gives you persistent NAND memory at close-to-memory speed (2.5G/sec write, 3G/sec read)
● Violin gives you 22TB flash memory (2G/sec)● By 2016 we can expect consumer harddisk with 60TB (Seagate)
● Most companies don't have many terabytes of real data that needs to be accessed 'at once'.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
MariaDB Foundation Overview
The Foundation is the new driver of the MariaDB project
Custodian of the code, Guardian of the community
Foundation can never to be controlled by a single entity or person
Foundation designed to be self-sustaining
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
MariaDB Foundation Details
A US 501c(3) not-for-profit8 founders and 2 sponsors so far (including all MySQL founders!)Open for more founders until FebruaryIn February the founders will the choose board etc.
In the interim we have:● Monty as CTO● Andrew Katz as COO and Legal Person (checking all the
paperwork)● Niall McCarthy as Business Development (signing up
founders)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
MariaDB Foundation Goals
That MariaDB be actively developed in the community and to:
● Increase adoption of MariaDB● Ensure sustainable high-quality efforts to build, test and
distribute MariaDB● Ensure that community patches are reviewed and adopted● Guarantee a community voice● Keep MariaDB compatible with MySQL● Maintain mariadb.org
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
MariaDB Foundation
More founders and sponsors are welcome!
If you care about the future of the MySQL ecosystem, please contact us andand ask how you can get involved!
Niall McCarthy [email protected] Widenius [email protected] Katz [email protected]