MySQL Cluster NoSQL Memcached API

MySQL Cluster: NoSQL Memcached API

•  Meet New Requirements: Add persistence, sharding, fault tolerance

•  Ease-of-Use: Standard Memcached libraries

•  Simplification: Eliminate cache-coherency & consolidate data tiers

•  Flexible Data Model: Schema / Schema-less storage

•  Flexible Access Patterns: SQL and NoSQL

•  Flexible Deployments: values stored in Cluster, Memcached or both

•  Key / Value Performance: Native access to MySQL Cluster

Copyright © 2012 Oracle Corp.

<Insert Picture Here>

Program Agenda

•  Overview, MySQL Cluster

•  Memcached API

•  Memcache Overview •  MySQL Cluster (NDB) & Memcache setup •  Sample Application

•  Resources to Get Started


MySQL Cluster

• Auto-Sharding, Multi-Master • ACID Compliant, OLTP + Real-Time Analytics

HIGH SCALE, READS + WRITES

• Shared nothing, no Single Point of Failure • Self Healing + On-Line Operations

99.999% AVAILABILITY

• Open Source + Commercial Editions • Commodity hardware + Management, Monitoring Toos LOW TCO

• Key/Value + Complex, Relational Queries • SQL + Memcached + Java + JPA + HTTP/REST + C++ SQL + NoSQL

• High Load, Real Time Performance • Predictable Low-Latency, Bounded Access Time REAL-TIME


MySQL Cluster 7.2 70x Faster JOINs

Fastest Ramp…Ever

8x Higher Per Node Performance

writes

NoSQL Memcached API

Geo-Distributed Clusters

Memcached API



Memcached Overview

Memcache Client

Application memcached

memcached

memcached memcache key friends:12389!

hash key to find data

hash key to pick server


Memcache Protocol Elements

• Key Up to 250 characters • Expire Time A number in seconds, up to 30 days •  Flags 32-bit number for application use • CAS ID 64-bit incrementing version number on value • Value Value stored for key; traditionally up to 1 MB.


Memcache Protocol Commands (1)

•  get key Fetch value for this key •  set key value Store this data •  add key value Store this data (strict insert) •  replace key value Store this data (strict update) •  cas key value cas-id Update with version number check


Memcache Protocol Commands (2)

•  append key value Append text to current value •  prepend key value Insert text before current value •  incr key amount Add to current stored integer value •  decr key amount Subtract from current stored value •  delete key Delete a key and value •  flush_all Delete all stored values on the server •  stats stat-name Fetch server statistics


Traditional architecture: Memcached & MySQL

MySQL Slave

more storage (on disk)!more powerful protocol (SQL)!

higher throughput !

lower latency!

mysql client

memcached Memcache

Client

Application


NDB & Memcache Architecture: Memcache protocol + NDB storage

MySQL Cluster

Data Node memcached

Memcache Client

Application

MySQL Cluster

Data Node

NDB Engine


Goals • Access stored data directly from memcache client •  Memcached perspective:

•  MySQL Cluster is a write-scalable, replicated data store •  with reliable in-memory storage, •  plus on-disk storage when data is too big for memory.

•  MySQL Cluster perspective: •  memcache is a high performance API •  providing easy access to in-memory data, •  plus an extra layer of caching when data is on disk.


Measured Latency memcachetest -t 2 -M 7000 -c 25000 !


Configuration Overview NDB Memcache


Anatomy of a Memcache key

user:1248

the prefix the database key


A key-prefix mapping

Memcache key

prefix Cluster Container Cache

Policy


key prefix Cluster Con-

tainer Cache Policy

Server Role

ID

A memcache server role


tainer Cache Policy


tainer Cache Policy


tainer Cache Policy


Getting Started MySQL Cluster and Memcached


“Cluster on a laptop”

• Configuration file cluster.ini • Management server ndb_mgmd • Data node ndbd • MySQL server mysqld • Memcached server memcached

(All of these are present in MySQL Cluster 7.2 distribution)

Cluster Configuration File

A single central file for the whole cluster Loaded by the management server on --initial And then stored in cache for future restarts Sections:

[MGM] [DB DEFAULT] [NDBD] [API]


[MGM] section: Management Server

[MGM]NodeId=1datadir=/Users/jdd/sandbox/ndbhostname=localhost


[DB DEFAULT] Section: data node parameters

[DB DEFAULT]NoOfReplicas=1TimeBetweenGlobalCheckpoints=12000HeartbeatIntervalDbApi=15000DataMemory=180MIndexMemory=60M


[NDBD] Sections: individual data node

[NDBD]NodeId=2


[API] Sections for MySQL & Memcached servers

[API][API][API][API][API][API][API]


Complete cluster.ini file [MGM]NodeId=1datadir=/Users/jdd/sandbox/ndbhostname=localhost[DB DEFAULT]NoOfReplicas=1TimeBetweenGlobalCheckpoints=12000HeartbeatIntervalDbApi=15000DataMemory=180MIndexMemory=60M[NDBD]NodeId=2[API][API] ...


Start the Management Server

% ndb_mgmd -f config-file-path


Start the Data Node

% ndbd


Configure & start mysqld (my.cnf file)

[mysqld]ndbcluster


Load NDB Memcache configuration tables

% cd /usr/local/mysql/share/memcache-api% mysql -u root < ndb_memcache_metadata.sql



Sample Application


"Twitter-like" Application

•  Two Tables •  authors •  tweets

•  Three Perl Functions •  Create a new author record •  Insert a new tweet •  Increment an author's tweet count

Required Perl Modules

% cpan install Cache::Memcached% cpan install UUID::Tiny


Sample Application

% mysql -u rootmysql> CREATE DATABASE demo; use demo;


Data Definition: Authors

CREATE TABLE authors ( author varchar(15) not null, date_created timestamp, tweets bigint unsigned not null default 0, followers bigint unsigned not null default 0, following bigint unsigned not null default 0, PRIMARY KEY (author)) ENGINE=ndb;


Data Definition: Tweets

CREATE TABLE tweets ( id varchar(36) primary key, author varchar(15), time_stamp timestamp, tweet varchar(140), index(time_stamp), index(author))ENGINE=ndb;


Configuration: Use the ndbmemcache config schema

mysql> use ndbmemcache; show tables;


INSERT into containers SET name = 'tweets_table' , db_schema = 'demo' , db_table = 'tweets', key_columns = 'id' , value_columns = 'time_stamp,author,tweet';

INSERT into key_prefixesSET key_prefix = 'tweet:', policy = 'ndb-only', container = 'tweets_table';

Configuration: Create container & key_prefixes



Two key_prefixes for authors table

•  newauthor: •  ADD an author record and set date_created •  tweetcount: •  INCR tweet count for author

INSERT into containers SET name = 'author_new' , db_schema = 'demo' , db_table = 'authors',key_columns = 'author' ,

value_columns = 'date_created';

INSERT into key_prefixesSET key_prefix = 'newauthor:', policy = 'ndb-only', container = 'author_new';

Configuration: New Author


INSERT into containers SET name = 'author_count', db_schema = 'demo', db_table = 'authors', key_columns = 'author', increment_column = 'tweets';

INSERT into key_prefixesSET key_prefix = 'tweetcount:', policy = 'ndb-only', container = 'author_count';

Configuration: Tweet Count


% cd /usr/local/mysql% bin/memcached -E lib/ndb_engine.so

Memcached: Start the server

17-Aug-2012 13:53:32 PDT NDB Memcache 5.5.25-ndb-7.2.7 ...Contacting primary management server (localhost:1186) ... Connected to "localhost:1186" as node id 4.Retrieved 6 key prefixes for server role "default_role".Server started with 4 threads.Priming the pump ... done [0.661 sec].


use Cache::Memcached;use UUID::Tiny;our $mc = new Cache::Memcached( { 'servers' => [ "localhost:11211" ]});

Perl: Startup Code


Perl: Add a new author

## Takes one parameter: authorsub new_author { my $author = shift; my $time = time(); my $key = "newauthor:$author"; $mc->add($key, $time);}


## Take two parameters: author and tweetsub insert_tweet { my $author = shift; my $tweet = shift; my $id = create_UUID_as_string(UUID_V4); my $time = time(); # Insert the tweet my $key = "tweet:$id"; my $value = "$time" ."\t". $author ."\t". $tweet; $mc->add($key, $value);}

Perl: Insert a Tweet


## Increment the author's tweet count.## Take one parameter: authorsub increment_tweetcount { my $author = shift; my $key = "tweetcount:$author"; $mc->incr($key, 1);}

Perl: Increment Tweet Count


Download MySQL Cluster + Memcached API

http://www.mysql.com/downloads/cluster/#downloads

Most secure, scalable MySQL Database, Development / Management / Monitoring Tools, backed by Oracle Premier Lifetime Support

Oracle Premier Support

Oracle Product Certifications/Integrations

MySQL Enterprise Security

MySQL Enterprise Scalability

MySQL Enterprise Backup

MySQL Enterprise Monitor/Query Analyzer

MySQL Workbench

MySQL Cluster CGE

MySQL Cluster

Manager

•  Straight from the Source •  Largest Team of MySQL Experts •  Direct Access to MySQL Support Engineers •  Backed by MySQL Developers •  Global 24/7/365, 29 languages •  Forward Compatible Hot Fixes •  MySQL Maintenance Releases •  Unlimited Incidents •  Knowledge Base •  MySQL Consultative Support

Oracle Premier Support for MySQL Rely on The Experts - Get Unique Benefits

"The MySQL support service has been essential in helping us with troubleshooting and providing recommendations for the production cluster, Thanks." -- Carlos Morales – Playfulplay.com

Only From Oracle

Getting Started

Learn More

Get MySQL Cluster 7.2 Review Documentation

Scaling Web Databases Guide www.mysql.com/cluster/

Download Today http://www.mysql.com/downloads/cluster/

Copyright 2011 Oracle Corporation 50

https://dev.mysql.com/doc/refman/5.5/en/mysql-cluster-ndbmemcache.html

Summary

Scale Web Services with 99.999% Availability

Native Memcached NoSQL Access

Try it out Today!

Copyright 2011 Oracle Corporation 51

MySQL Cluster Architecture

Data Nodes

Node Group 1

F1

F3

F3

F1

Nod

e 1

Nod

e 2

Node Group 2

F2

F4

F4

F2

Nod

e 3

Nod

e 4

Application Nodes

Cluster Mgmt

Cluster Mgmt

REST JPA

MySQL Cluster - Extreme Resilience

Data Nodes

Node Group 1

F1

F3

F3

F1

Nod

e 1

Nod

e 2

Node Group 2

F2

F4

F4

F2

Nod

e 3

Nod

e 4

Application Nodes

Cluster Mgmt

Cluster Mgmt

REST JPA

MySQL Cluster NoSQL Memcached API

Technology

memcached copyright

value value value

database key copyright

key value performance

memcache key user

key set key valuestore

mysql cluster perspective

memory data