The State of Apache HBase - SK Planetreadme.skplanet.com/wp-content/uploads/key_hbase... · Chair of the Apache HBase PMC* ... Apache Drill & HBase! Thank You! [email protected]. TODO

The State of Apache HBaseMichael Stack

Michael Stack <stack@{apache.org,cloudera.com}>

* Project Management Committee

● Chair of the Apache HBase PMC*● Caretaker/Janitor

● Member of the Hadoop PMC● Engineer at Cloudera in SF

What is it?

”... is an open source, distributed, scalable, consistent, low latency, non-relational, random access database”

Built on Apache

● Hadoop core:– Distributed file system (HDFS)– MapReduce

● HBase persists all data to HDFS● Uses Apache ZooKeeper

– Cluster coordination● Goal: “Billions of rows X millions of columns on

clusters of 'commodity hardware'”

ZK HDFS

App MR

Inspiration

A Google Technology described in a 2006 paper, Bigtable: A Distributed Storage System for Structured Data by Chang et al.?

First commit...

commit 454a9dbe046194f8eef3dddc3e5942910dd5b7a1Author: Douglass Cutting <[email protected]>Date: Tue Apr 3 20:34:28 2007 +0000

HADOOP-1045. Add contrib/hbase, a BigTable-like online database.

DISTRIBUTIONS

When to use it?

BIG Data

scalE!

Low-latency, online, random read/writes

*Very like Google Bigtable model only different nomenclature

Datamodel*

DataModel: A Bigtable!

● 0-N Bigtable(s)● Rows x Column Families● Column Families

● Has columns● CF prefix and qualifier

● e.g. attribute:mimetype

Datamodel: Regions● Table splits into “regions”

● Automatically as table grows● Region has contiguous rows

● [startRow, endRow)

DataModel: Sorted & Versioned● All is byte []

● No native 'types'● Schema-less (NoSQL)

● All is SORTED● Rows in byte-lexographical order● Columns sorted along row

● VERSIONED● Cells are “versioned”● 3D (timestamp)

Datamodel: Strongly consistent● Row modifications are atomic

● Even if thousands of columns on a row● Favors consistency over availability

● “Designing applications to cope with concurrency anomalies in their data is very error-prone, time-consuming, and ultimately not worth the performance gains” -- F1: A Distributed SQL Database That Scales

Architecture: Physical

HDFS NameNodesHBase Masters

ZooKeeper Quorum

Slave Boxes (DN + RS)

Cluster is made of a Master and Slaves Nodes

Features

•Classes to MapReduce HBase tables•Query predicate push down via server side filters •Coprocessors (stored procedures/triggers)•Extensible jruby-based (JIRB) shell•Replication•Security

– Table/Column Family– Kerberos Authentication, ACLs

What to expect• Writes:

– 1-3ms, 1k-20k writes/sec per node

• Reads:– 0-3ms cached, 10-30ms disk– 10-40k reads / second / node from cache– > if SSD

• Cell size• 0-3MB preferred

• Column-orientated so wide tables are OK• Sparsely populated rows OK

Who uses it?

In Production

● OLTP & Batch● Messages○ 1B+ users○ Tens of PBs (compressed)○ Thousands of machines, Pods of ~200

● ODS/Real-time monitoring/Timeseries○ Dual write two clusters○ Critical eyes and ears

● All on AWS● 5 production clusters and growing● Mix of SSD and SATA● Billions of page views per month

Users

● Long time HBase user● Two clusters of 1k nodes each

○ Master-Master replicating● Separate low-latency cluster

○ Up to 1M reads a second

Cassini● Ebay item search indexing● 600M active items in HBase tables● 1.4TB of data processed each day● 400M puts to HBase each day● 250M search metrics per day● Two datacenters● Growing clusters...

– 500->1k

Deploy types• Multitenant multifarious feature storeo a.k.a dumping groundo Stumbleupon, Y!, SalesForce

• Reconciliation storeo ebay

• Timeserieso SalesForce, FB ODS

• Lots-o-entities storeo Flurry, genomeo Lots-o-entities BLOBs, FB Messages

Who runs the project?

Diverse team*

* http://hbase.apache.org/team-list.html

COMMITTERS!

Preferably ALIVE!

Dev Rate

# of commits

Total Files 2021Total Lines of Code 832122Total Commits 6615 (~ 3/day)Authors 39

(https://www.ohloh.net/p/hbase)

JIRA: 2008-2013

Commits/Month Over Time (0.94/trunk)

HBase Today

•Release every month• Each more stable•& more performant•Some features…• Wire compatible between releases

•Currently at 0.94.13

http://www.flickr.com/photos/sysli/3026288256/sizes/o/in/photostream/

● 0.96.0– Released October 19th, 2013– 18months in the making

● >2000 fixes

Big Themes● Stability● Operability

– Insight, tools● Scalability● Evolvability

● Pluggable Compression– Smarter triggers

● Hadoop1 AND Hadoop2● Smarter Region Balancer● Region Assignment Hardened● Coprocessors

– More hooks

Sampler

http://www.flickr.com/photos/allspaw/5815258929/sizes/o/in/photostream/

•System tables• Filesystem•Up in zookeeper•Over the wire

Namespaces• Grouping of tables

– Like database in mysql

• System/User– hbase:meta

• Quota• Coming

– Security by namespace– Grouping on cluster by namespace

And more...• X-row (in-region) Transactions• Query tracing• New UI• Online Merge• Hardened Replication• Off-heap bucket cache• Metrics2o Radical revamp

• By the end of the year• Rolling upgrade from 0.96.0

• In-line Cell-tags– Security++

● ACL down to the Cell-level● Cell-level visibility labels

• Reverse Scan

●HBase 1.0.0●Reining in the 99th percentiles

●Multi-WAL●Speculative replica reads

●More support for multi-tenancy●Off-heap

HBase 2014

HBaseEcosystem

OpenTSDB● Timeseries● Store, index and serve metrics at large scale● Make data easily accessible and graphable

HaeinsaHaeinsa 란 무엇인가 ?

Is a linearly scalable multi-row, multi-table transaction library for HBase. Haeinsa uses two-phase locking and optimistic concurrency control for implementing transaction. The isolation level of transaction is serializable.

● Inspired by Google Percolator● VCNC

Chasm

How to make it easier writing applications against HBase?

Frameworks: Kiji.org• Entity-centric, simple modelo Types, complex, compound types.

• Each cell is schema versioned

• Works across MR & REST, etc.

• Machine-learning libs

• Examples, tutorials

• Production users

• Open-source

Frameworks: CDK

• APIs providing Dataset abstraction – get/put/delete API in AVRO objects

• Highlights: – Supports multiple components

● flume, morphlines, hive, crunch, hcat – Types using Avro and parquet formats– Manages schema evolution

• Open source by Cloudera

– http://cloudera.github.io/cdk/docs/current

● Client-embedded JDBC driver○ Connection conn =

DriverManager.getConnection("jdbc:phoenix:localhost");

● Alternate HBase Client API (SQL)● Fast!

○ Exploits HBase Coprocessors/Filters○ Types○ Aggregations○ Skip scans○ Secondary indices

Beyond...● Hadoop Family evolving, growing● No longer just Batch

– Real-time– Streaming

● October Apache Hadoop 2.0 release an inflection point– O'Reilly Strata + Hadoop World NYC 2013

● Coming out party● New distributions● Enterprise

Beyond: No longer just batch● YARN

● Distributed scheduling● Resource management● More than just MR on the cluster● Arbitrary Apps

● Hive speedup Tez/Stinger● Storm

● “Streaming” Hadoop● Storm on YARN

Beyond: No longer just batch

● Apache● Apache

● Cluster management● Cloudera Impala

● Scalable low-latency SQL query● HDFS (& HBase)

● Apache Drill● & HBase!

Thank [email protected]

TODO

● DBA: R (read), W (write), C (create), X (execute), A (admin). ● cell-level security. Every cell in an Accumulo store can have a label, stored effectively as part of

the key, which is used to determine whether a value is visible to a given subject or not. The label is not an ACL, it is a different way of expressing security policy.

● A label instead turns this on its head and describes the sensitivity of the information to a decision engine that then figures out if the subject is authorized to view data of that sensitivity based on (potentially, many) factors.

● Then, as of HBASE-7662, HBase can store into and apply ACLs from cell tags, extending the current HBase ACL model down to the cell.

● Finally, we have also contributed transparent server side encryption, as HBASE-7544, for additional assurance against accidental leakage of data at rest, which is at this time an HBase-only feature.

● Auto-manages partitioning● Storage machinery in the RS● I like the Latency/Throughput/Read/Write axis in Nick