Developing II: Advanced Document Design

Advanced Document Design

J Chris AndersonMobile

When considering how to model data for a given application, you should…•Think of a logical container for the data•Think of how data groups together

You may notice that…•From a software development standpoint, this maps better to the way you structure the data•It’s a more natural way of representing the entities the application will be handling

Modeling Data

Example of A User Profile

• Key selection: start with the username

• Use this as a common prefix for related data

• When building user’s session information, fetch these items

• Create related documents by prefixing with the same username

• Can extend this concept further

• Other data records for this user, building out object graph, etc.

Example: Data Profile for a User

{ “_id”: “auser_profile”, “user_id”: 7778 “password”: “a1004cdcaa3191b7”, ”common_name”: ”Alan User”, ”nicknames”: [”Al”, ”Buddy”], "sign_up_timestamp": 1224612317, "last_login_timestamp": 1245613101}

{ “_id”: “auser_friends”, “friends”: [ “joe”, “ingenthr”, “toru” ]}

blog example

Modeling a Blog

6

CommentComment

PostPost

CommentComment

CommentComment

CommentComment

Blog post structure:•Main content itself•User to blogs is a 1:N relationship•Set of comments, also a 1:N relationship

Representing Objects● Simply treat an object as a JSON document

● Could also represent it with serialized objects from the higher level language

● Determine the key for an object using the class name (or type) of the object and an unique ID

Representing Object Lists● Denormalization● Save an array of object IDs in this list

Representing Blog Data in Couchbase

7

• Three entities (for this simple example)• Users• Hold the user’s profile, enough to instantiate the needed data for a session

• Blog lists• A simple list of blogs written by a given user– semi normalized

• Blogs• The blogs authored by a given user

Modeling the Example Blog Data

8

User ObjectKey: ’JChris'

JSON{ “_id” : “JChris”, “nid” : 1, “name” : “Chris” “password” : “sldkfjslkdfj”}

Blog Data in Couchbase

9

Blog ObjectKey: ’blog_spoke_at_couchconf'

JSON{ “_id” :

“blog_spoke_at_couchconf”, “user_id” : 1 “content-md” : “blah de bla

h”}

UserBlog ListKey: ’JChris_BlogList'

JSON{ “_id” : “Player1_Plantlist”, “blogtitles” : [“spoke_at_couchconf”, “(didn’t)_miss_sxsw”, “next_to_couch_st”]}

● No need to “ALTER TABLE”

● Add new “fields” all objects at any time– Specify default value for missing fields– Increased development speed

● Using JSON for data objects● This will allow future capabilities with Couchbase Server 2.0● Offers the ability to query and analyze arbitrary fields with views

Schema-less Modeling of Data

10

Creating a new blog post

// Create the new blogBlog aBlog = new Blog(”ready_for_lunch");// do more

cbclient.set(aBlog.getId(), aBlog);

// Update the user’s bloglistUser theUser = user.fetch(cbclient.get(username);

// Add the plant to the playertheUser.addPost(aBlog);

// Store the player's new plant listcbclient.set(username + “_bloglist", theUser.getBlogList());

Modifying Blog Data in Couchbase

11

• User profile• Main pointer into the user data:• Blog entries• Badge settings, like a twitter badge

• Blog posts• Contains the blogs themselves• Blog comments• Comments from other users

Adding Comments to a Blog

{

“_id”: “jchris_Hello_World”,

“author”: “jchris”,

“type”: “post”

“title”: “Hello World”,

“format”: “markdown”,

“body”: “Hello from [Couchbase](http://couchbase.com).”,

“html”: “<p>Hello from <a href=\“http: …

“comments”: [

[ “format”: “markdown”, “body”:”Awesome post!” ],

[ “format”: “markdown”, “body”:”Like it.” ]

]

}

Blog Document Sample

Blog Document Sample, Broken up

{

“_id”: “jchris_Hello_World”,

“author”: “jchris”,

“type”: “post”

“title”: “Hello World”,

“format”: “markdown”,

“body”: “Hello from [Couchbase](http://couchbase.com).”,

“html”: “<p>Hello from <a href=\“http: …

“comments”: [

“comment1_jchris_Hello_world”

]

}

{“_id”: “comment1_jchris_Hello_World”,“format”: “markdown”, “body”:”Awesome post!” }

You can imagine how to take this to a threaded list

Threaded Comments

Blog First comment Reply to

comment

More Comments

ListList

Advantages:• Only fetch the data when you need it• For example, rendering part of the page, jQuery style• Spread the data (and thus the load) across the entire cluster

Couchbase Server 2.0:Querying and Aggregation

with Views

1616

• Simulate the way federated systems work:

• Examples: checking accounts, credit card transactions

• Create a record per transaction

• Leverage views to reconcile the results of the transaction

• If results don’t reconcile, there is a missing transaction or a flaw in the business logic

Exchanging Virtual Currency

{ “From”: “matt”, “to”: “jchris”, “coins”: 30 }

{ “From”: “matt”, “to”: “perry”, “coins”: 30 }

{ “From”: “jchris”, “to”: “perry”, “coins”: 30 }

{ “From”: “matt”, “to”: “jchris”, “coins”: 30 }

{ “From”: “matt”, “to”: “perry”, “coins”: 30 }

{ “From”: “jchris”, “to”: “perry”, “coins”: 30 }

Exchanging Virtual Currency

{ “From”: “matt”, “to”: “jchris”, “coins”: 30 } { “From”: “matt”, “to”: “perry”, “coins”: 30 } { “From”: “jchris”, “to”: “perry”, “coins”: 30 }

Mapperfunction(transaction) { emit(transaction.from, transaction.amount *-1); emit (transaction.to, transaction.amount); }

Reducefunction(keys, values) { return sum(values);}

Query balance view with key == user, get the balance for

the user.

Query the sum of the entire view, value should be 0.

Q: How can I implement transactions across multiple documents?

A: Transactions in this distributed system can be re-imagined with views. (example to follow…)

Q: Can I write more complex aggregation logic? For example apply a transform or a filter when reducing?

A: While the built in reduce functions are often enough and highly tuned, Couchbase Server can execute arbitrary javascript functions.

Common Questions

• Player items• Find all the weapons owned by an individual player• Leader board view• Showing who has the highest level in the system

Demo: Game Simulator Views

Questions?