Top Banner
©2013 DataStax Confidential. Do not distribute without consent. @chbatey Christopher Batey Use case & Lessons learned: Internet Television
26
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Paris Day Cassandra: Use case

©2013 DataStax Confidential. Do not distribute without consent.

@chbateyChristopher Batey

Use case & Lessons learned: Internet Television

Page 2: Paris Day Cassandra: Use case

@chbatey

Who am I?•Built a a lot of systems with Apache Cassandra at Sky•Work on a testing library for Cassandra•Help out Cassandra users•Twitter: @chbatey

Page 3: Paris Day Cassandra: Use case

@chbatey

Agenda• Migrating a system designed for 5k users to 5million

users- Many different uses of Cassandra- One example data model• Deployment• Lessons learned• A live migration (mysql to Cassandra)

Page 4: Paris Day Cassandra: Use case

@chbatey

Internet Television• Functionality- Manage users - millions of them- Manage devices - even more- Entitlements - Who can watch what?- Product catalogs- Event logging for billing and analytics

Page 5: Paris Day Cassandra: Use case

@chbatey

Internet Television• Non-functional requirements- Don’t kill other systems- Very spiky - Game of Thrones, Champions Leage- Increasing user base - doubling per year- Multi DC an absolute must

Page 6: Paris Day Cassandra: Use case

@chbatey

5k Users

Page 7: Paris Day Cassandra: Use case

@chbatey

100k+ Users

Page 8: Paris Day Cassandra: Use case

@chbatey

Champions League match?

Page 9: Paris Day Cassandra: Use case

@chbatey

Uh oh :(

Page 10: Paris Day Cassandra: Use case

@chbatey

A pile of cats

Page 11: Paris Day Cassandra: Use case

@chbatey

The solution

Micro(ish)services +

Page 12: Paris Day Cassandra: Use case

@chbatey

Strangulation

Page 13: Paris Day Cassandra: Use case

@chbatey

The new

Page 14: Paris Day Cassandra: Use case

@chbatey

Audit log• Very heavy write - many events per user interaction with

the service

Page 15: Paris Day Cassandra: Use case

@chbatey

Modelling the traditional wayCREATE TABLE customer_events( customer_id text, staff_name text, time timeuuid, event_type text, store_name text, PRIMARY KEY (customer_id)); CREATE TABLE store( name text, location text, store_type text, PRIMARY KEY (store_name)); CREATE TABLE staff( name text, favourite_colour text, job_title text, PRIMARY KEY (name));

Page 16: Paris Day Cassandra: Use case

@chbatey

Your model should look like your queries

Page 17: Paris Day Cassandra: Use case

Modelling in CassandraCREATE TABLE customer_events(

customer_id text, staff_id text, time timeuuid, store_type text, event_type text, tags map<text, text>, PRIMARY KEY ((customer_id), time));

Partition Key

Clustering Column(s)

Page 18: Paris Day Cassandra: Use case

@chbatey

Lesson #1: State kills scalability• There are only two places to put state in a scalable

systems:- The client- Cassandra

Page 19: Paris Day Cassandra: Use case

@chbatey

Lesson #2: Test & measure• Only you know your work loads- Try different schemas, record the performance• Cassandra stress is your friend

Page 20: Paris Day Cassandra: Use case

@chbatey

Lesson #3: Use the DataStax docs

Page 21: Paris Day Cassandra: Use case

@chbatey

Lesson #4: Say no to thrift

http://exponential.io/blog/2015/01/08/cassandra-terminology/

Page 22: Paris Day Cassandra: Use case

@chbatey

Lesson #5: DevOps• Know the hardware you are deploying Cassandra on• No network storage• AWS => Ephemeral storage

Page 23: Paris Day Cassandra: Use case

@chbatey

Lesson #6: Run multi node clusters

Page 24: Paris Day Cassandra: Use case

@chbatey

Live migration (mysql —> Cassandra)• Double write - make the writes idempotent

Page 25: Paris Day Cassandra: Use case

@chbatey

Summary• Cassandra allows you to run at a scale not possible with

other DBs• The cost? Denormalising + duplicating

Page 26: Paris Day Cassandra: Use case

@chbatey

Questions?• Think of any questions later? @chbatey