SUSE: Software Defined Storage

Software Defined StoragePowered by Ceph

Olaf KirchDirector SUSE Linux Enterprise, SUSE R&D

okir@suse.com

2

Current Enterprise Data Storage Market

More data to store•business needs•more data driven processes•more applications•e-commerce

More data to store•business needs•more data driven processes•more applications•e-commerce

Bigger data to store•richer media types•presentations, images, video

Bigger data to store•richer media types•presentations, images, video

For longer•regulations / compliance needs•business intelligence needs

For longer•regulations / compliance needs•business intelligence needs

2000 2014

3

While you think about Storage...

• ... can you also make it‒ more scalable

‒ more fault tolerant

‒ more flexible?

• Sure! For here or to go?

4

Today's Storage Arrays

• Limits:‒ Tightly controlled

environment

‒ Limited scalability

‒ Few options

‒ Only certain approved drives

‒ Constrained number of disk slots

‒ Few memory variations

‒ Only very few networking choices

‒ Typically fixed controller and CPU

• Benefits:‒ Reasonably easy to

understand

‒ Long-term experience and “gut instincts”

‒ Somewhat deterministic behavior and pricing

5

What about Better File Systems?

• Layered on top of your block storage, providing‒ scalability

‒ fault tolerance

• Most of the time, it's an either-or decision‒ separation of data and metadata (pNFS, glusterfs)

‒ clustering (ocfs2, gfs2)

‒ ... they require special drivers in your favorite OS

‒ ... and they all want to talk to a storage array

• Recent evolution‒ HDFS (underlying FS of Hadoop)

Software Defined Block Storage?

© Nhobgood (CC-BY-SA)

What is Ceph?

8

From 10,000 Meters

[1] As per 2014 OpenStack user survey

• Open Source Distributed Storage solution

• Most popular choice of distributed storage for

OpenStack[1]

• Lots of goodies‒ Distributed Object Storage

‒ Redundancy

‒ Efficient Scale-Out

‒ Extensible

‒ Can be built on commodity hardware

9

From 1,000 meters

10

Not for the Faint of Heart

• At the core of Ceph is a PhD Thesis‒ http://ceph.com/papers/weil-crush-sc06.pdf

• Goals:‒ no bottlenecks

‒ no single point of failure

11

Several Ingredients

• Distributed‒ Coarse grained partitioning of storage supports policy based

mapping (don't put all copies of my data in one rack)

‒ Topology map and Rules allow clients to “compute” the exact location of any storage object

• Redundancy‒ Achieved by data replication

• Flexibility‒ Multiple Storage “Pools” can be defined with different

parameters

Some Basic Concepts

13

For a Moment, Zooming to Atom Level

FS

Disk

OSD Object Storage Daemon

File System (btrfs, xfs)

Physical Disk

● OSDs serve storage objects to clients● Peer to perform replication and recovery

14

Put Several of These in One Node

FS

Disk

OSD

FS

Disk

OSD

FS

Disk

OSD

FS

Disk

OSD

FS

Disk

OSD

FS

Disk

OSD

15

Mix In a Few Monitor Nodes

M • Monitors are the brain cells of the cluster‒ Cluster Membership‒ Consensus for Distributed Decision Making

• Do not serve stored objects to clients

16

Voilà, a Small RADOS Cluster

M MM

17

Linux Host

RADOS Block Device

M MM

M

krbd librados

18

RADOS Block Device

• Disk images are striped across (parts of) the cluster

• Supports‒ Snapshot and rollback

‒ COW cloning

‒ Thin provisioning

19

RADOS Block Device: Placement

M MM

M

20

Ceph in Action: Reading Data

M MM

M

Reads can be serviced by any of the replicas (parallel reads

improve thruput)

21

Ceph in Action: Writing

M MM

M

Writes go to one OSD, which then propagates the

changes to other replicas

Self-Healing

23

Self-Healing

M MM

M

Monitors detect a dead OSD

24

Self-Healing

M MM

M

Monitors allocate other OSDs and update mapping

25

Self-Healing

M MM

M

Monitors initiate recovery

26

Self-Healing

M MM

M

Future writes update the new

replica

Why am I telling you this?

28

Because we think Ceph is Great!

29

By 2018, open-source storage will gain 20% of the market share, up from less than 1% in 2013

(Gartner)

30

Enterprise Data Capacity Utilization (Percent)

50-60% of Enterprise Data

20-25%

15-20%

1-3%

Tier 0Ultra HighPerformance

Tier 1High-value, OLTP, Revenue Generating

Tier 2Backup/Recovery,Reference Data, Bulk Data

Tier 3Object, Archive,Compliance Archive,Long-term Retention

Source: Horison Information Strategies - Fred Moore

31

SUSE Enterprise Storage Market

LOW FUNCTIONALITY

HIGHFUNCTIONALITY

ObjectStorage

ArchiveStorage

DataBackup

Video Audio

BigData

DataAnalytics

OLTP

CRMERP

Email

HPC

ComplianceArchiveCAPACITY

OPTIMIZED

PERFORMANCEOPTIMIZED

DRTarget

Initial Target MarketInitial Target Market

BulkStorage

VM-Aware

In Closing

33

Summary: Why Ceph?

• Can be scaled arbitrarily

‒ No central bottleneck “master” servers

• Low operational cost

‒ Automation

‒ Commodity hardware

• Can move data close to application

• Redundancy through data replication

‒ Self-healing

‒ No need for RAID

• APIs and cloud integration for self-service

‒ Software defined storage

Thank you.

34

For questions: okir@suse.comPlease visit us aswww.suse.com

Corporate HeadquartersMaxfeldstrasse 590409 NurembergGermany

+49 911 740 53 0 (Worldwide)www.suse.com

Join us on:www.opensuse.org

36

Unpublished Work of SUSE. All Rights Reserved.This work is an unpublished work and contains confidential, proprietary and trade secret information of SUSE. Access to this work is restricted to SUSE employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of SUSE. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.

General DisclaimerThis document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. SUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for SUSE products remains at the sole discretion of SUSE. Further, SUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All SUSE marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.