1 CS 525 Advanced Distributed Systems Spring 2014 Indranil Gupta (Indy) Feb 11, 2014 Key-value/NoSQL Stores Lecture 7 2014, I. Gupta Based mostly on •Cassandra NoSQL presentation •Cassandra 1.0 documentation at datastax.com •Cassandra Apache project wiki •HBase
CS 525 Advanced Distributed Systems Spring 2014. Indranil Gupta (Indy) Feb 11, 2014 Key-value/NoSQL Stores Lecture 7. Based mostly on Cassandra NoSQL presentation Cassandra 1.0 documentation at datastax.com Cassandra Apache project wiki HBase. 2014, I . Gupta. Cassandra. - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
CS 525 Advanced Distributed Systems
Spring 2014
Indranil Gupta (Indy)
Feb 11, 2014
Key-value/NoSQL Stores
Lecture 7
2014, I. Gupta
Based mostly on •Cassandra NoSQL presentation•Cassandra 1.0 documentation at datastax.com•Cassandra Apache project wiki•HBase
2
Cassandra
• Originally designed at Facebook
• Open-sourced
• Some of its myriad users:
• With this many users, one would think– Its design is very complex
– Let’s find out!
3
Why Key-value Store?
• (Business) Key -> Value
• (twitter.com) tweet id -> information about tweet
• (amazon.com) item number -> information about it
• (kayak.com) Flight number -> information about flight, e.g., availability
• (yourbank.com) Account number -> information about it
• Search is usually built on top of a key-value store
4
Isn’t that just a database? • Yes, sort of
• Relational Databases (RDBMSs) have been around for ages
• MySQL is the most popular among them
• Data stored in tables
• Schema-based, i.e., structured tables
• Queried using SQL (Structured Query Language)
SQL queries: SELECT user_id from users WHERE username = “jbellis”
Example’s Source
5
Mismatch with today’s workloads
• Data: Large and unstructured
• Lots of random reads and writes
• Foreign keys rarely needed
• Need– Speed
– Avoid Single point of Failure (SPoF)
– Low TCO (Total cost of operation) and fewer sysadmins
– Incremental Scalability
– Scale out, not up: use more machines that are off the shelf (COTS), not more powerful machines
6
CAP Theorem
• Proposed by Eric Brewer (Berkeley)
• Subsequently proved by Gilbert and Lynch
• In a distributed system you can satisfy at
most 2 out of the 3 guarantees1. Consistency: all nodes see same data at any time, or
reads return latest written value
2. Availability: the system allows operations all the time
3. Partition-tolerance: the system continues to work in spite of network partitions
• After recovery from failure, or upon bootup (HRegionServer/HMaster)
– Replay any stale logs (use timestamps to find out where the database is w.r.t. the logs)
– Replay: add edits to the MemStore
• Keeps one HLog per HRegionServer rather than per region
– Avoids many concurrent writes, which on the local file system may involve many disk seeks
32
Cross-data center replication
HLog
Zookeeper is actually a file system for control information1. /hbase/replication/state2. /hbase/replication/peers /<peer cluster number>3. /hbase/replication/rs/<hlog>
33
Wrap UpWrap Up
• Key-value stores and NoSQL trade off between Availability and Consistency, while providing partition-tolerance
• Presentations and reviews start this Thursday– Presenters: please email me slides at least 24 hours before
your presentation time
– Everyone: please read instructions on website
– If you have not signed up for a presentation slot, you’ll need to do all reviews.