1 www.snia.org OSD TWG Mike Mesnier January 2003 Object-based Storage 101 SNIA
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
www.snia.org
OSD TWG
Mike Mesnier
January 2003
Object-based Storage
101
SN
IA
2
www.snia.org
OSD TWG
Storage Trends/Facts
• Storage is moving to the network
– Servers connect to consolidated NAS or SAN storage
• Storage and networking fabrics are converging
– iSCSI, iFCP, RDMA/TCP, …
• Storage is becoming more autonomous
– Standards for richer interfaces and protocols
– More functionality in the storage device
– Research focusing on intelligent storage
• Self-aware, self-managing, self-configuring
3
www.snia.org
OSD TWG
SwitchedSwitched
FabricFabric
Compute and storageCompute and storage
platforms are tailoredplatforms are tailored
for specific functions.for specific functions.
DB Servers
File Servers
Web Servers
Load Balancers
SAN Storage
Building Blocks
Storage Bricks
Compute Bricks
Tonight’s Talk
Evolving Data Center
Fire Walls
4
www.snia.org
OSD TWG
• Imagine a storage brick that is:
– Always available and reliable
– Self-securing
– Self-managed
– Infinitely scalable (i.e., stackable)
Object-based storage is a step closer…
Storage Utopia !
5
www.snia.org
OSD TWG
• " Storage today
• Emerging solutions
• Object-based storage
• SNIA activity
Agenda
6
www.snia.org
OSD TWG
Storage App.
HOST
…
Direct Attached
Storage
CLIENT CLIENT
Network
…
File System
Network Attached
Storage
Storage App
HOSTStorage App
HOST
Network
…
Storage Area
Network
How did these evolve?
Local Storage Shared Data Shared Capacity
Architectures Today
7
www.snia.org
OSD TWG
• DAS – physically secure and simple– Not scalable
– Limited connectivity for capacity sharing
– No data sharing
• SAN – scalable capacity sharing– Limited (coarse) security through switch
– No data sharing
• NAS – secure sharing (data or capacity)
– Limited scalability
The Trade-offs
8
www.snia.org
OSD TWG
• DAS and SAN provide raw block storage– No model for the stored data (e.g., files)
– No support for data sharing (e.g., concurrency control)
• Storage applications built using DAS or SAN– File systems, databases, video servers, etc.
– App. implements a data model (e.g., a file)
– May implement direct data sharing (e.g., via clustering)
• NAS is an application for indirectly sharing data– Servers export local file systems over the network
– Clients share data by sharing files
The Usage Models
9
www.snia.org
OSD TWG
• Data centers need to share data.– Backup, HSM
– Clustering for scalability (e.g., file server and db)
• Sharing data directly on the SAN is difficult.– Concurrency control, lock management, versioning
– Distributed security
• Sharing data through NAS is easy– But imposes limits in scalability
• Single point of failure & bottleneck
Challenge: Data Sharing
10
www.snia.org
OSD TWG
• Storage today
• "Emerging solutions
• Object-based storage
• SNIA activity
Agenda
11
www.snia.org
OSD TWG
• Virtualization (NAS & SAN)
– Aggregates heterogeneous NAS and SAN devices
• Clustering the file system
– Mitigates file server bottleneck
• Giving clients direct access to storage devices
– File servers share block metadata with clients
– Eliminates file server bottleneck
• Changing the device interface
– From blocks to objects
Emerging Solutions
NAS ClientsNAS Clients
e.g. Webe.g. Web
serversservers
SAN
Islands
NAS
Islands
SAN
Virtualization
NAS
VirtualizationClients need direct
access to remove
bottleneck.
Network
FILES
+
Clustering
BLOCKS
1st Generation File Server
OSD TWG
13
www.snia.org
OSD TWG
ServersServers
Block-Based Storage Block-Based Storage
MANAGEMENT
Eth switchTrusted SAN
DA
TA
Clients Clients
METADATA
Must be trusted
Difficult to directly share
2nd Generation File Server
14
www.snia.org
OSD TWG
• Storage today
• Problems and partial solutions
• " Object-based storage
• SNIA activity
Agenda
15
www.snia.org
OSD TWG
!"#$% & '
()*+,-.' /0 &
1234 56.7 % &
Objects can be self-describing!
• An object comprises– Application data (e.g., file, record)
– Device-managed metadata (e.g., block allocation)
– User-accessible attributes (e.g., access times)
• Objects have file-like methods for access– Open, close, read, write, get/set attributes
– Commands are authorized
• Object-based storage devices– Disk drive, appliance, controllers
Object Storage
16
www.snia.org
OSD TWG
Object Storage Model
Block Interface
Storage Device
Block I/O Manager
Object Interface
Applications
File System
User Component
File System
Storage Component
System Call Interface
CPU
Applications
File System
User Component
System Call Interface
CPU
Storage Device
Block I/O Manager
File System
Storage Component
17
www.snia.org
OSD TWG
So What’s the Real Value of
Objects?
18
www.snia.org
OSD TWG
The Value of Objects
• Better security via capabilities– Each object can have its own security domain
– All I/O is authorized by the device
• Easier to share data– Files and records can be stored as objects
– Low-level metadata managed by device
• Opportunities for intelligence– Attribute-based learning for resource allocation
• Better caching, pre-fetching and staging of data
– Self-configuring storage w/ continuous reorganization• Layout objects to best serve client requests
19
www.snia.org
OSD TWG
• Separates policy from enforcement
– Storage managers set policy
– Storage devices enforce the policy
• Prevents unauthorized access
• Minimizes interaction with storage manager
• Minimizes state kept on device
– For better scalability, recovery and cost
Value #1: Security
20
www.snia.org
OSD TWG
Security Types
• Types of security– Authentication – “you are who you say”
– Authorization – “you have permission”
– Integrity – “data is not corrupted/modified”
– Privacy – “data is not seen”
• TWG is considering two scenarios– Channel is trusted
– Channel is not trusted
OSD handles all but authentication.
21
www.snia.org
OSD TWG
Preventable Attacks
• Snooping or modification of commands and data
• Unauthorized access via modified capability
• Delay and replay attacks
• Guards against these attacks, respectively:
– Transport or app-level encryption for privacy
– Transport or app-level digests for integrity
– Cryptographically secure capability
– Nonce (timestamp) attached to each command
• Requires loosely synchronized clocks
• Only needed when channel is not trusted
22
www.snia.org
OSD TWG
• Less metadata to keep coherent
– No block allocation or free block lists
– Shorter “lists” to manage
• Objects may contain aggregation metadata
– Leads to better scalability
• Backup and HSM
– No need to co-locate w/ application (e.g., file system)
– Only backup necessary objects (not entire volume)
Value #2: Data Sharing
OSD is really a file system less the naming
23
www.snia.org
OSD TWG
3rd Generation File Server
Managers
Object-based Storage Devices
MANAGEMENT
Eth switchSAN
Clients
SECRETSECRET
KEYKEY
SECRETSECRET
KEYKEY
SECRETSECRET
KEYKEY
Access Request
DA
TA
Intelligent Device
Space
Management
Backup/Recovery
QoS via attributes
Security
Validate CapabilityValidate Capability
24
www.snia.org
OSD TWG
• Objects can have rich attributes– Timestamps, accounting information, QoS parameters, group and
user information, client-specific usage patterns
• Many opportunities for policy-based management– E.g, attributes may describe backup and QoS policies
• Attributes may also provide “information gain”– An object’s actions may be correlated with its attributes
• E.g., Any object written within the last 24 hours will be read up at 4a.m. the following day (i.e., for backup)
• Can devices actually learn??– This is an active research topic
– OSD facilitates a further investigation
– Big questions: which attributes really matter?
Value #3: Intelligence
25
www.snia.org
OSD TWG
• Storage today
• Emerging solutions
• Object-based storage
• " SNIA activity
Agenda
26
www.snia.org
OSD TWG
• Charter and Program of Work:
– Requirements for OSD
– Transport independent definition of OSD
– SCSI Standard for OSD
– White papers & industry demonstrations
• ~100 members and over 30 organizations
– academia, industry, National Laboratories andstartups.
The SNIA TWG for OSD
27
www.snia.org
OSD TWG
• Establishing the commands – v1 DONE
• Establishing the attributes – v1 DONE
• Security architecture – v1 DONE
• How should we identify and locate objects?
• How to maintain integrity through failures?
• Should we support transactional semantics?
• Management of hundreds or thousands of OSDs.
Work Items in the TWG
28
www.snia.org
OSD TWG
OSD FunctionsSecurity
• Authorization
• Integrity
• Privacy
Attributes
• Aid to file systems
• Hints to device
• QoS and Priorities
• Logging & Statistics
Basic Protocol
– Read
– Write
– Create
– Delete
– Open
– Close
– Get
– Set
– Append
– Clear
Basic I/O
Space Mgmt
Session “hints”
Attributes
Other cmds
29
www.snia.org
OSD TWG
• A big challenge in the enterprise is data sharing
– Must be secure and scalable
• Objects complement existing technologies
– Add security, low-overhead data sharing and intelligence
– Enable the 3rd generation file server
• OSD is more active now than ever
– SNIA has become the focal point
– Will complete v1 SCSI standard 1st half of this year.
Summary
30
www.snia.org
OSD TWG
Call to Action
• How must your file system change to support OSD?
• How will OSD change your backup story?
• See the latest OSD standard at T10
• Understand our plans for other transports
• Get involved in the OSD TWG
– Architecture
– Standards efforts
– Industry demonstrations
31
www.snia.org
OSD TWG
• Academic research
– www.pdl.cmu.edu
– www.dtc.umn.edu
– csl.cse.ucsc.edu/obsd.shtml
• Standards work
– www.snia.org/osd
– www.nsic.org/nasd
– www.t10.org/scsi-3.htm (see OSD)
• Industry research & development
– www.intel.com/labs/storage/osd
• Download OSD Reference Code
– www.haifa.il.ibm.com/storage.html
– www.lustre.org
– www.panasas.com
Further References
32
www.snia.org
OSD TWG
Our Leaders
• Mike Mesnier, Julian Satran (co-Chairs)
• Applications – Erik Riedel
• Education – Tom Ruwart
• Management – Ken Samarra
• Security – Michael Factor
Contact us to get involved!
Related Documents
