RICK KRUEGER

RICK KRUEGER DAVE VALENTINE @dataogre dataogre.com

linkedin.com/in/premierapps

@ingeniousSQL ingeniousSQL.com

linkedin.com/in/ingenioussql

RULES OF ENGAGEMENT:NOSQL IS SQL SERVER’S ALLY

Why Rules of Engagement?

“Directives issued by competent military authority that delineate the circumstances and limitations under which United States forces will initiate and/or continue combat engagement with other forces encountered” - http://ra.defense.gov (JP-104)

Why Rules of Engagement?

“Directives issued by competent military authority that delineate the circumstances and limitations under which United States forces will initiate and/or continue combat engagement with other forces encountered” - http://ra.defense.gov (JP-104)

Why Rules of Engagement?

“Directives issued by competent military authority that Delineate the circumstances and limitations under which United States forces will initiate and/or continue combat engagement with other forces encountered” - http://ra.defense.gov (JP-104)

Why Rules of Engagement?

“Directives issued by competent military authority that delineate the circumstances and limitations under which United States forces will

initiate and/or continue Combat engagement with other forces encountered” - http://ra.defense.gov (JP-104)

AGENDA

NoSQL Family Key Value Column Store Document Databases Graph Databases

Polyglot Persistence Eventual Consistency Wrap Up Questions & Answers

Key Value

SimpleMy Car123

C-4140-9564-5B282124

U.S. Air Force photo

Key Value

Simple

Fast

My Car123

C-4140-9564-5B282124

U.S. Air Force photo

Key Value

Simple

FastScalable

My Car123

C-4140-9564-5B282124

U.S. Air Force photo

Key Value

Simple

FastScalableCheap

My Car123

C-4140-9564-5B282124

U.S. Air Force photo

Key Value

Simple

FastScalableCheapKey Structure

My Car123

C-4140-9564-5B282124

U.S. Air Force photo

We have to wait how long?

Key ValueCouchbase

Adidas, ADP, BMWLinux, Windows, OS Xhttp://www.couchbase.com/

RiakYammer, Best Buy, GithubLinux, OS Xhttp://basho.com/riak/

RedisTwitter, Craigslist, flickrLinux, OS Xhttp://redis.io/

Key Value – When to use

Keys value pairsWeb ComponentsDevice Check-in

Key Value: The Need For Speed BEFORE

100K

AFTER

100K

Cach

e

Column Stores

DoD photo by Petty Officer 1st Class Benjamin D. Olvey, U.S. Navy. (Released)

Column Stores

123455 Car: 626 Year: 2008 Name: Jen Insurance: St Farm

ID Name Insurance Car Year Warrant

y123455 Jen St Farm 626 2008 NULL12356 Bob

12357 John Geico Speed 3 2013 Yes

123456 Name: Bob

123457 Car: Speed 3 Year: 2013 Name: John Insurance: Geico 10

11

12

Type: Oil

Type: Tires

Type: Wipers

Warranty: Yes

ID Type CarID10 Oil 1235711 Tires 12357

12 Wipers 12357

Person_Details Car_Details

Column Stores

ScalableSimple is FastColumn AggregationComplex = Map ReduceComplex can be Slow

DoD photo by Petty Officer 1st Class Benjamin D. Olvey, U.S. Navy. (Released)

Column StoresScalableSimple is FastColumn AggregationComplex = Map ReduceComplex can be Slow

DoD photo by Petty Officer 1st Class Benjamin D. Olvey, U.S. Navy. (Released)

Column StoresScalableSimple is FastColumn AggregationComplex = Map ReduceComplex can be Slow

DoD photo by Petty Officer 1st Class Benjamin D. Olvey, U.S. Navy. (Released)

Column StoresScalableSimple is FastColumn AggregationComplex = Map ReduceComplex can be Slow

DoD photo by Petty Officer 1st Class Benjamin D. Olvey, U.S. Navy. (Released)

Column StoresScalableSimple is FastColumn AggregationComplex = Map ReduceComplex can be Slow

DoD photo by Petty Officer 1st Class Benjamin D. Olvey, U.S. Navy. (Released)

Column StoresHBase

eBay, BlackBerry, ExpediaLinux, OS X, Windows **http://hbase.apache.org/

CassandraFacebook, Shazam, TwitterLinux, OS X, Windowshttp://cassandra.apache.org/

** Additional software is required

HD

Insigh

t

Column Stores: Do I need it…

Unstable Schema

Nullable Columns

Collections

http://www.youtube.com/watch?v=7hfpQMu5xJ0

Document Databases

"That's so crazy that it just might work!"

Document Databases: Planning

http://www.blueangels.navy.mil/

Document Databases: Schemaless?

Document Databases: Not Relational

Trims Table411 1790 Base 118412 1790 John Cooper Works 208413 1790 S 172414 1791 Base 121

Models Table1790 Clubman1791 Countryman1792 Coupe1793 Roadster

Document Databases: NormalizedJSON:

JavaScript Object Notation

Document Databases: Denormalized

Document Databases: Natural

Document Databases: OptionsmongoDB

Craigslist, Foursquare, Shutterfly, IntuitWindows, Linux, and OS Xhttp://www.mongodb.org/

CouchDB (CouchBase)HootSuite, VimeoWindows, Linux, and OS Xhttp://couchdb.apache.org/

RavenDBMSNBC.COM, JetBrainsWindows and Linuxhttp://ravendb.net/

Document Databases

RavenDB Management Studio

Document Databases

CouchDB Futon

You Might Need A Document Database…if your application is document based (eg Content Management, Blogging, User Data Management), or the data isn’t tabular or structured…if you want to archive data, and do not want to deal with schema changes in the archived table…if you are looking for an easier upgrade path or have constant schema churn…if you find yourself using SQL antipatterns, like EAV (Entity-attribute-value)…if programmer friendliness is a big concern

Graph DatabasesRelationships

Nodes PropertiesEdges

RecordsDataPK / FK

2: (Make: Mini, Model: Clubman)

1: (Name: Rick, DOB:20120505)

OWNS:(VIN: 12BA126)

DoD photo by Staff Sgt. Samuel BendetU.S. Air Force (Released)

Graph Databases

Graph Databases

1 | Rick

2 | Dave

3 | Mini

4 | Yugo

Owns

Wants

Wants

Owns

6 | 626

Want

s

5 | Glen

Owns

Wants

http://jalopnik.com/365474/how-about-a-yugo-for-the-24-hours-of-lemons

7 | Brandon

9 | Bob

11 | Nick

8 | S4

13 | A6

10 | M4

12 | Neon

Owns

OwnsWants

Owns

Wants

Owns

Wan

ts

Graph Databases

https://www.facebook.com/about/graphsearch

Graph DatabasesNeo4j

Adobe, Cisco, T-MobileLinux, OS X, Windowshttp://www.neo4j.org/

AllegroGraphNASA, Kodak, PfizerLinux, OS X, Windowshttp://www.franz.com/agraph/allegrograph/

InfiniteGraphCIA, DODLinux, OS X, Windowshttp://www.objectivity.com/infinitegraph

Graph Databases

Graph Databases

Graph Databases

Graph Databases – When to use…

Network

Relationships

Geo

Polyglot Persistence"This is space! Course, we’re just in the beginning part of space, we-we haven’t even got to *outer* space yet!“

– Armageddon (1998)

Polyglot Persistence: Write-Only

Polyglot Persistence: Write Only Graph Database? Key Value? Column Store? Document Database?

Polyglot Persistence: Schema Upgrade

Polyglot Persistence: Archiving

Polyglot Persistence: Cache

OLTP Warehouse

Other Data

Nightly

Nightly

Polyglot Persistence: Cache

Warehouse

Other Data

Version 1

Version 2

Version 3

Polyglot Persistence: 1000 Words

Consistency

Consistency: We want it ALL Speed Availability Scale

Consistency Latency

Consistency: Definitions

N – the number of Nodes (or replicas) that store the dataW – the minimum number of nodes that must acknowledge the Write before considered completeR – the minimum number of Read nodes (or replicas) queried for a read operation

Consistency: Distributed

Replica 1

Replica 2

Replica 3

Consistency: Strong

W+R > N == Strong Consistency

N = 2W = 2 R = 1

Consistency: Strong

W+R > N == Strong Consistency

N = 2W = 1 R = 2

Consistency: Weak

N = 2W = 1 R = 1

W+R <= N == Weak Consistency

Consistency LatencyStrong Weak Low High

Synchronous ReplicationHigh Safety MirroringSynchronous AlwaysOn Availability GroupAsynchronous ReplicationHigh Performance MirroringAsynchronous AlwaysOn Availability Group

Active Secondaries: Readable Secondary Replicas“In many circumstances, data latency between a primary database and the corresponding secondary database is only a few seconds.” -http://msdn.microsoft.com/en-us/library/ff878253.aspx

Consistency: SQL Scale Out

Consistency: Cost

Google search result count = 30 resulted in 20% loss

Amazon, 100ms delays perceptible to users

Speed matters!

Rules of Engagement

Key Value

Column Store

Document

Database

Graph

RDBMS

90% of

Data

Data Size / Scalability

Data

Co

mpl

exity

Remember

Pick the store that matches the shape of your data

Don’t use SQL Server to build Rube Goldberg Machines

If anyone thinks that NoSQL means no planning, they couldn’t be more wrong

Stay informed: NoSQLWeekly.com

Questions?

RICK KRUEGER DAVE VALENTINE @dataogre dataogre.com

linkedin.com/in/premierapps

@ingeniousSQL

ingeniousSQL.com

linkedin.com/in/ingenioussql