C* Summit EU 2013: Hardware Agnostic: Cassandra on Raspberry Pi

#CASSANDRAEU CASSANDRASUMMITEU

Hardware Agnostic: Cassandra on Raspberry Pi

Andy Cobley | Lecturer, University of Dundee, Scotland


*  Cassandra is hardware agnostic *  So why not run it on a Raspberry Pi ? *  How hard can it be ? *  What can we do with it once it works?

What we will discuss today…


*  Andy Cobley *  Program Director, MSc in Data Science and Business Intelligence *  School of Computing *  University of Dundee *  Twitter: @andycobley

Who Am I ?


*  Single chip Linux computer *  500 Meg ram *  Boots off an SD card *  Ethernet port *  (graphics and all you need for a general purpose computer)

Whats a Raspberry Pi ?


Pi with pound coin


*  Also 4 node cluster.

And, here’s one for real


*  Cassandra is designed to be fast, fast at writing, fast at reading. *  This laptop with one instance of Cassandra will do 12,000 write

operations *  Raspberry Pi will do 200 !

The Bad News


*  Running a external USB drive is actually worse ! *  Probably be hardware feature

More bad news !


Raspberry Pi Schematic


*  Oracle Java vs OpenJDK

And then there’s Java!


*  Raspbian is Debian for the PI *  Uses the Hard floating point accelerator *  Much faster than Debian *  Current official Oracle JDK won’t run on it !

And Raspbian


*  http://www.oracle.com/technetwork/java/embedded/downloads/javase/index.html

*  Java SE Embedded version 6 *  Cassandra might prefer 6 (or 7 for Cassandra 2) *  But *  https://blogs.oracle.com/henrik/entry/oracle_releases_jdk_for_linux *  Preview at: *  https://jdk8.java.net/fxarmpreview/

Oracle java


*  And then it turns out:

Actually not much difference in performance

Hard vs Soft Float


*  Cassandra uses compression for performance *  Started in version 1.0

2x-‐4x reduc8on in data size 25-‐35% performance improvement on reads 5-‐10% performance improvement on writes

The Problem with compression


*  Three types:

Google Snappy Compressor (Faster read/writes) DeflateCompressor (Java zip, slower , beOer compression)

*  Snappy Compression not available on Pi

(requires na8ve methods, so someone might get it to work!)

Compression types


*  Cassandra 1.2 (and 2) also has lz4 compression *  Which is good news !

Compression


*  Startup script allocates memory *  Calculates based on number of processors *  Pi reports Zero processors ! *  Boom ! *  Now fixed

And the startup script


*  In Cassandra-env.sh *  JVM_OPTS="$JVM_OPTS -

Djava.rmi.server.hostname=192.168.1.15” *  Or else nodetool will not work between nodes

JMX Config


*  C* 1.22. added UseCondCardMark as a JVM Opt *  "for better lock handling especially on hotspot with multicore

processor” *  In cassandra-env.sh

#if [ "$JVM_VERSION" \> "1.7" ] ; then # JVM_OPTS="$JVM_OPTS -‐XX:+UseCondCardMark" #fi

JVM OPT UseCondCardMark



*  We’ve forgotten one thing *  The Pi cost £25 *  You can power 4 from USB hub (no need for a power supply on

each one) *  So:

The Good News !


So, have a 64 node computer for £2000

University of Southhampton


*  32 node Beowolf cluster: *  Joshua Kiepert, Boise University

Or this


Or this Hadoop Cluster from LinkedIn

hOp://prac8calcloudcompu8ng.com/post/53996976003/hadoop-‐running-‐on-‐a-‐14-‐chip-‐raspberry-‐pi-‐cluster


*  Adding nodes adds performance *  Adding nodes adds replicas of data *  BUT *  Make sure your ring is balanced, *  Pi’s don’t like to be unbalanced.

Adding nodes is good


*  Vnodes (in 1.2) would be very nice *  However at this point I haven’t got 1.2 on Pi running on a cluster *  As for Cassandra 2, see later

Vnodes


Performance with 3/4 nodes


Performance with 5/6 nodes


*  ./stress -d 192.168.1.10,192.168.1.11,192.168.1.12 -o insert -I DeflateCompressor

*  Note: nodes to use *  You will get different performance if you insert to less nodes than

you have in your ring

Stress test commands


*  On Debian, you can free memory from the graphics chip

Cd /boot sudo cp start.elf start.elf.old sudo cp arm224_start.elf to start.elf reboot

Getting more memory


*  Under Rasbian *  Run with a monitor plugged for the first time *  Set options for screen memory *  Perhaps disable boot to GUI

Getting more Memory


*  I prefer static network addresses *  Edit /etc/network/interfaces iface eth0 inet sta8c address 192.168.1.41 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.254

Network address


*  Make a master SD card *  Copy it ! *  Make sure the master version has no data on it. *  Consider ”Puppet” (though I don’t use it)

Multiple nodes


*  See https://github.com/acobley/CassandraStartup *  Put the file in /etc/init.d *  update-rc.d cassandra defaults

Starting as a service


*  So for £200 we get an 8 node C* cluster *  It can be reconfigured, blown away, stress tested and generally

abused *  We can simulate data racks, data centers and I hope even long

network delays. *  Hopefully our students will use these clusters

Pi is for teaching


*  We know C* can be configured to be aware of:

Network racks Data Centers

*  We know we can have replicas are stored across these racks *  How can we play with this cheaply

C* is network aware


Proposed teaching tool

10mbs Hubb

Noise injec8on

Switch 2

Switch 1


*  What about the Linux tc command *  Lets look again at the diagram

TC ?


*  What we can’t do *  Recommended bandwidth is 1000 Mbit/s (Gigabit) or greater. *  Bind the Thrift interface (listen_address) to a specific NIC (Network

Interface Card). *  Bind the RPC server interface (rpc_address) to another NIC.

Network

hOp://www.datastax.com/docs/1.2/cluster_architecture/cluster_planning


*  Internode compression currently uses Snappy *  So turn it off in conf file:

internode_compression: none

What about Cassandra 2.0


*  Some bad news

*  So need to tune it : *  See John Berryman’s blog: *  http://www.opensourceconnections.com/2013/08/31/building-the-

perfect-cassandra-test-environment/

How does C* 2 run on a PI


*  Cassandra wouldn’t run on a PI *  It does now. *  Running it on a Pi shook out some Cassandra bugs *  You can run it in a secure lab

Pi is discovery


*  Most important, this was pure Geeky Fun

Pi is for fun


*  Data Science: *  http://www.computing.dundee.ac.uk/study/postgrad/

degreedetails.asp?17

Obligatory Plug



*  Raspberry Pi is cheap *  C* needs some work to run on it *  You can make clusters cheaply for experimentation *  It’s fun !

What we discussed today…


THANK YOU

C* Summit EU 2013: Hardware Agnostic: Cassandra on Raspberry Pi

Technology

pi requiresna8vemethods

pi reports

raspberry pi schematic

pi cost

instance of cassandra

multiple nodes

compression types

beoer compression