NoSQL - An introduction to CouchDB

CouchDB A Database for the Web

Jonathan Weiss

Who am I?

Working for Peritor in Berlin, Germany

Written, maintain, or involved in

Webistrano

Capistrano

SimplyStored

Happening

The great fire of London

http://github.com/jweiss

@jweiss

2

Scalarium

Amazon EC2 Cluster Management

Auto-config

Self-Healing

Auto-Scaling

One-click-deployment

www.scalarium.com

3

Database Requirements

High Availability

Easy Replication

Clustering

Robustness

Short Recovery Time

4

5

CouchDB

Build for the Web

Scales

Replication built-in

Embracing offline

Flexible schema – document DB

6

7

”CouchDB is built of the Web“ Jacob Kaplan-Moss

Web Technologies

HTTP the access protocol

JavaScript the query language

JSON the storage format

8

JSON Document

{ "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } }

9

JSON Document


10

JSON Document


11

No Tables or Namespaces

12

No Tables or Namespaces

13

Manual Namespacing


14

Interacting with CouchDB

15

PUT /dbname/ID

JSON

HTTP Client


16

JSON

GET /dbname/ID

HTTP Client


17

DELETE /dbname/ID HTTP Client

18

Views

Design Document { "id": "_design/hats”, "_rev": "431212AB4”, "language": "javascript”, "views": { "all": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } }

19


20

Document ID –

prefixed by “_design/”


21

Hash of Views


22

Hash of Views Every view has map &

reduce function

Map

function(doc) { if (doc.headware) { for (var hat in doc.headware) { emit(hat, 1); } } }

23

Map


24

Passed every document in the DB

Map


25

Inspects & Decides

Map


26

Emits result for index

Reduce

function(keys, values, rereduce) { return sum(values); }

27

Reduce


28

Passed map result (or partial reduce result)

Reduce


29

Aggregates (count, sum, average, …)

Example Map Result

Map functions are similar to SQL indices

30

ID KEY VALUE

51ABFA211 Cap 1

ABC123456 Cappy 1

BCCD12CBB Helmet 1

BCCD12CBB Sombrero 1

Sorted by the key

Key can also be an array

Value can be complex JSON and/or reference to other document

Query a view

31

GET /dbname/_design/hats/_view/all

HTTP Client

{"total_rows":348,"offset":0,"rows”:[ {"id":"A","key":"A","value":1}, {"id":"B","key":"B","value":1}, ]}

Query a view

32

GET /dbname/_design/hats/_view/all?include_docs=true

HTTP Client

View Query

Filter by

key=ABC123

startkey=123 & endkey=9

limit=100

descending=true

group=true

reduce=true

Include_docs=true

33

SQL vs. JavaScript

34

Vs.

Using Couch from PHP

Several options available

PHPillow: http://arbitracker.org/phpillow.html (LGPL 3)

PHP Object_Freezer: https://github.com/sebastianbergmann/php-object-freezer/tree (BSD)

PHP On Couch: http://github.com/dready92/PHP-on-Couch/tree/master (GPLv2 or v3)

PHP CouchDB Extension: http://www.topdog.za.net/php_couchdb_extension (PHP License 3.0)

Sag for CouchDB: http://www.saggingcouch.com/ (Apache License 2.0)

35

36

37

38

39

SimplyStored

CouchDB convenience Layer for Ruby

Models & Associations

Validations

Callbacks

Dynamic finder

S3 attachments

Paranoid delete

ActiveModel compliant

40

BSD-licensed on http://github.com/peritor/simply_stored

On top of CouchPotato, CouchRest & RestClient

Many more

Or just build your own using HTTP!

41

Database Requirements

High Availability

Easy Replication

Clustering

Robustness

Short Recovery Time

42

Replication

XXXXX

43

B-Tree

Photo by Mathias Meyer

B-Tree

Append only

Concurrency (MVCC)

Crash resistant

Hot backups

Compaction

44

Replication

45

CouchDB Replication

46

POST /_replicate

POST /_replicate

Eventually consistent & conflict resolution

Load Balancing

47

HTTP Client HTTP Load Balancer

Replication

Caching

48

HTTP Client HTTP Cache Varnish Apache …

Multi-Master

49

BigCouch

50

Clustered CouchDB:

Many CouchDBs appear as one

Modeled after Amazon Dynamo

Scalability like Cassandra or Riak

github.com/cloudant/bigcouch

BigCouch

51

Q – No. of partitions

N – No. of replicas

R – Read quorum

W – Write quorum

Various

CouchApps

Validations

Filtered replication

Changes feed

List functions

Futon

Geo

Fulltext-Search with embedded Lucene

Different experimental View-Server 52

© Peritor GmbH - Alle Rechte vorbehalten

Peritor GmbH Blücherstr. 22, Hof III Aufgang 6 10961 Berlin

Tel.: +49 (0)30 69 20 09 84 0 Fax: +49 (0)30 69 20 09 84 9

Internet: www.peritor.com E-Mail: [email protected]

Q&A

NoSQL - An introduction to CouchDB

Documents

NoSQL - An introduction to CouchDB