NoSQL - An introduction to CouchDB

CouchDB A Database for the Web

Jonathan Weiss

Who am I?

Working for Peritor in Berlin, Germany

Written, maintain, or involved in

  Webistrano

  Capistrano

  SimplyStored

  Happening

  The great fire of London

http://github.com/jweiss

@jweiss

2

Scalarium

Amazon EC2 Cluster Management

  Auto-config

  Self-Healing

  Auto-Scaling

  One-click-deployment

www.scalarium.com

3

Database Requirements

High Availability

Easy Replication

Clustering

Robustness

Short Recovery Time

4

5

CouchDB

Build for the Web

Scales

Replication built-in

Embracing offline

Flexible schema – document DB

6

7

”CouchDB is built of the Web“ Jacob Kaplan-Moss

Web Technologies

HTTP the access protocol

JavaScript the query language

JSON the storage format

8

JSON Document

{ "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } }

9

JSON Document

{ "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } }

10

JSON Document

{ "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } }

11

No Tables or Namespaces

12

No Tables or Namespaces

13

Manual Namespacing

{ "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } }

14

Interacting with CouchDB

15

PUT /dbname/ID

JSON

HTTP Client

Interacting with CouchDB

16

JSON

GET /dbname/ID

HTTP Client

Interacting with CouchDB

17

DELETE /dbname/ID HTTP Client

18

Views

Design Document { "id": "_design/hats”, "_rev": "431212AB4”, "language": "javascript”, "views": { "all": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } }

19

Design Document { "id": "_design/hats”, "_rev": "431212AB4”, "language": "javascript”, "views": { "all": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } }

20

Document ID –

prefixed by “_design/”

Design Document { "id": "_design/hats”, "_rev": "431212AB4”, "language": "javascript”, "views": { "all": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } }

21

Hash of Views

Design Document { "id": "_design/hats”, "_rev": "431212AB4”, "language": "javascript”, "views": { "all": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } }

22

Hash of Views Every view has map &

reduce function

Map

function(doc) { if (doc.headware) { for (var hat in doc.headware) { emit(hat, 1); } } }

23

Map

function(doc) { if (doc.headware) { for (var hat in doc.headware) { emit(hat, 1); } } }

24

Passed every document in the DB

Map

function(doc) { if (doc.headware) { for (var hat in doc.headware) { emit(hat, 1); } } }

25

Inspects & Decides

Map

function(doc) { if (doc.headware) { for (var hat in doc.headware) { emit(hat, 1); } } }

26

Emits result for index

Reduce

function(keys, values, rereduce) { return sum(values); }

27

Reduce

function(keys, values, rereduce) { return sum(values); }

28

Passed map result (or partial reduce result)

Reduce

function(keys, values, rereduce) { return sum(values); }

29

Aggregates (count, sum, average, …)

Example Map Result

Map functions are similar to SQL indices

30

ID KEY VALUE

51ABFA211 Cap 1

ABC123456 Cappy 1

BCCD12CBB Helmet 1

BCCD12CBB Sombrero 1

Sorted by the key

Key can also be an array

Value can be complex JSON and/or reference to other document

Query a view

31

GET /dbname/_design/hats/_view/all

HTTP Client

{"total_rows":348,"offset":0,"rows”:[ {"id":"A","key":"A","value":1}, {"id":"B","key":"B","value":1}, ]}

Query a view

32

GET /dbname/_design/hats/_view/all?include_docs=true

HTTP Client

View Query

Filter by

  key=ABC123

  startkey=123 & endkey=9

  limit=100

  descending=true

  group=true

  reduce=true

  Include_docs=true

33

SQL vs. JavaScript

34

Vs.

Using Couch from PHP

Several options available

  PHPillow: http://arbitracker.org/phpillow.html (LGPL 3)

  PHP Object_Freezer: https://github.com/sebastianbergmann/php-object-freezer/tree (BSD)

  PHP On Couch: http://github.com/dready92/PHP-on-Couch/tree/master (GPLv2 or v3)

  PHP CouchDB Extension: http://www.topdog.za.net/php_couchdb_extension (PHP License 3.0)

  Sag for CouchDB: http://www.saggingcouch.com/ (Apache License 2.0)

35

36

37

38

39

SimplyStored

CouchDB convenience Layer for Ruby

  Models & Associations

  Validations

  Callbacks

  Dynamic finder

  S3 attachments

  Paranoid delete

  ActiveModel compliant

40

BSD-licensed on http://github.com/peritor/simply_stored

On top of CouchPotato, CouchRest & RestClient

Many more

Or just build your own using HTTP!

41

Database Requirements

High Availability

Easy Replication

Clustering

Robustness

Short Recovery Time

42

Replication

XXXXX

43

B-Tree

Photo by Mathias Meyer

B-Tree

Append only

Concurrency (MVCC)

Crash resistant

Hot backups

Compaction

44

Replication

45

CouchDB Replication

46

POST /_replicate

POST /_replicate

Eventually consistent & conflict resolution

Load Balancing

47

HTTP Client HTTP Load Balancer

Replication

Caching

48

HTTP Client HTTP Cache Varnish Apache …

Multi-Master

49

BigCouch

50

  Clustered CouchDB:

Many CouchDBs appear as one

 Modeled after Amazon Dynamo

  Scalability like Cassandra or Riak

  github.com/cloudant/bigcouch

BigCouch

51

Q – No. of partitions

N – No. of replicas

R – Read quorum

W – Write quorum

Various

CouchApps

Validations

Filtered replication

Changes feed

List functions

Futon

Geo

Fulltext-Search with embedded Lucene

Different experimental View-Server 52

© Peritor GmbH - Alle Rechte vorbehalten

Peritor GmbH Blücherstr. 22, Hof III Aufgang 6 10961 Berlin

Tel.: +49 (0)30 69 20 09 84 0 Fax: +49 (0)30 69 20 09 84 9

Internet: www.peritor.com E-Mail: [email protected]

Q&A