SONAS Architecture and Implementation

Front coverDraft Document for Review November 1, 2010 9:32 am SG24-7875-00

IBM Scale Out Network Attached Storage Architecture, Planning and Implementation BasicsLearn to setup and customize the IBM Scale Out NAS Details hardware and software architecture Includes daily administration scenarios

Mary Lovelace Vincent Boucher Shradha Nayak Curtis Neal Lukasz Razmuk John Sing John Tarella

ibm.com/redbooks

Draft Document for Review November 1, 2010 9:32 am

7875edno.fm

International Technical Support Organization SONAS Architecture and Implementation November 2010

SG24-7875-00

7875edno.fm


Note: Before using this information and the product it supports, read the information in Notices on page xiii.

First Edition (November 2010) This edition applies to IBM Scale Out Network Attached Storage V1.1.1. This document created or updated on November 1, 2010.

Copyright International Business Machines Corporation 2010. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.


7875edno.fm

iii

7875edno.fm


iv

SONAS Architecture and Implementation


7875TOC.fm

ContentsNotices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv The team who wrote this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Now you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii Chapter 1. Introduction to Scale Out File Network Attached Storage . . . . . . . . . . . . . . 1 1.1 Marketplace requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Understanding I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.1 File I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.2 Block I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2.3 Network Attached Storage (NAS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3 Scale Out Network Attached Storage (SONAS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3.1 SONAS architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3.2 SONAS scale out capability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.3.3 SONAS software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.3.4 High availability design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.4 SONAS architectural concepts and principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.4.1 Create, write, and read files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.4.2 Creating and writing a file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.4.3 Scale out more performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.4.4 Reading a file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 1.4.5 Scale out parallelism and high concurrency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.4.6 Manage storage centrally and automatically. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 1.4.7 SONAS logical storage pools for tiered storage . . . . . . . . . . . . . . . . . . . . . . . . . . 24 1.4.8 SONAS Software central policy engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 1.4.9 High performance SONAS scan engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 1.4.10 High performance physical data movement for ILM / HSM. . . . . . . . . . . . . . . . . 31 1.4.11 Hierarchical storage management, backup/restore to external storage . . . . . . . 33 1.4.12 Requirements for high performance external HSM and backup restore . . . . . . . 34 1.4.13 SONAS high performance HSM using Tivoli Storage Manager . . . . . . . . . . . . . 34 1.4.14 SONAS high performance backup/restore using Tivoli Storage Manager . . . . . 35 1.4.15 SONAS and Tivoli Storage Manager integration in more detail . . . . . . . . . . . . . 36 1.4.16 Summary - lifecycle of a file using SONAS Software . . . . . . . . . . . . . . . . . . . . . 39 1.4.17 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Chapter 2. Hardware architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Interface nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.2 Storage nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.3 Management nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Internal Infiniband switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Internal private Ethernet switch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.3 External Ethernet switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.4 External ports - 1 GbE / 10 GbE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Storage pods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Copyright IBM Corp. 2010. All rights reserved.

41 42 43 45 45 47 47 48 49 49 50 v

7875TOC.fm


2.3.1 SONAS storage controller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.2 SONAS storage expansion unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Connection between components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Interface node connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.2 Storage node connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.3 Management node connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.4 Internal POD connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.5 Data Infiniband network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.6 Management ethernet network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.7 Connection to the external customer network. . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Different SONAS configurations available . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Rack types - how to choose correct rack for your solution . . . . . . . . . . . . . . . . . . 2.5.2 Drive types - how to choose between different drive options . . . . . . . . . . . . . . . . 2.5.3 External ports - 1 GbE / 10 GbE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 SONAS with XIV Storage overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.1 Differences between SONAS with XIV and standard SONAS system . . . . . . . . . 2.6.2 SONAS with XIV configuration overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.3 SONAS base rack configuration when used with XIV storage . . . . . . . . . . . . . . . 2.6.4 SONAS with XIV configuration and component considerations . . . . . . . . . . . . . .

51 53 53 53 55 56 57 58 59 60 61 61 66 67 68 68 69 70 70

Chapter 3. Software architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 3.1 SONAS Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 3.2 SONAS data access layer - file access protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 3.2.1 File export protocols: CIFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 3.2.2 File export protocols: NFS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 3.2.3 File export protocols: FTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 3.2.4 File export protocols: HTTPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 3.2.5 SONAS Locks and Oplocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 3.3 SONAS Cluster Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 3.3.1 Introduction to the SONAS Cluster Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 3.3.2 Principles of SONAS workload allocation to interface nodes . . . . . . . . . . . . . . . . 81 3.3.3 Principles of interface node failover and failback . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.3.4 Principles of storage node failover and failback . . . . . . . . . . . . . . . . . . . . . . . . . . 84 3.3.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 3.3.6 SONAS Cluster Manager manages multi-platform concurrent file access . . . . . . 86 3.3.7 Distributed metadata manager for concurrent access and locking . . . . . . . . . . . . 88 3.3.8 SONAS Cluster Manager components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 3.4 SONAS authentication and authorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 3.4.1 SONAS authentication concepts and flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 3.5 Data repository layer - SONAS file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 3.5.1 SONAS file system scalability and maximum sizes . . . . . . . . . . . . . . . . . . . . . . . 97 3.5.2 Introduction to SONAS File System parallel clustered architecture . . . . . . . . . . . 97 3.5.3 SONAS File system performance and scalability . . . . . . . . . . . . . . . . . . . . . . . . . 98 3.6 SONAS data management services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 3.6.1 SONAS - Using the central policy engine and automatic tiered storage. . . . . . . 107 3.6.2 Using and configuring Tivoli Storage Manager HSM with SONAS basics . . . . . 111 3.7 SONAS resiliency using snapshots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 3.7.1 Integration with Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 3.8 SONAS resiliency using asynchronous replication . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 3.9 SONAS and Tivoli Storage Manager integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 3.9.1 General TSM and SONAS guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 3.9.2 Basic SONAS to TSM setup procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 3.9.3 TSM software licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

vi



7875TOC.fm

3.9.4 How to protect a SONAS files without TSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.10 SONAS system management services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.10.1 Management GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.10.2 Health Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.10.3 Command Line Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.10.4 External notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.11 Grouping concepts in SONAS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.11.1 Node grouping and TSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.11.2 Node grouping and async replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.12 Summary - SONAS Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.12.1 SONAS goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 4. Networking considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Review of network attached storage concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.1 File systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.2 Redirecting I/O over the network to a NAS device . . . . . . . . . . . . . . . . . . . . . . . 4.1.3 Network file system protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.4 Domain Name Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.5 Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Domain Name Server as used by SONAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Domain Name Server configuration recommendations . . . . . . . . . . . . . . . . . . . 4.2.2 Domain Name Server balances incoming workload . . . . . . . . . . . . . . . . . . . . . . 4.2.3 Interface node failover / failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Bonding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Bonding modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.2 Monitoring bonded ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Network groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Implementation networking considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.1 Network interface names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.2 Virtual Local Area Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.3 IP address ranges for internal connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.4 Use of Network Address Translation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.5 Management node as NTP server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.6 Maximum Transmission Unit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.7 Considerations and restrictions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 The impact of network latency on throughput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 5. SONAS policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Creating and managing policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 The SCAN engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 SONAS CLI policy commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 SONAS policy best practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.1 Cron jobs considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.2 Policy rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.3 Peered policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.4 Tiered policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.5 HSM policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.6 Policy triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.7 Weight expressions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.8 Migration filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.9 General considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Policy creation and execution walktrough . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 Create storage pool using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

124 125 126 128 129 130 131 132 135 135 138 139 140 140 140 141 143 143 143 144 145 146 148 149 149 149 151 151 151 152 152 153 153 153 153 157 158 158 160 162 162 164 165 165 166 166 167 167 167 168 168

Contents

vii

7875TOC.fm


5.4.2 5.4.3 5.4.4 5.4.5

Create storage pool using the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Create and apply policies using the GUI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Create and apply policies using the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Testing policy execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

170 171 173 175 177 178 178 180 181 181 181 182 182 183 184 185 188 188 188 189 193 193 194 194 195 200 211 212 213 213 215 216 216 217 217 218 220 224 227 228 228 229 234 235 235 236 236 237 238 238

Chapter 6. Backup and recovery, availability and resiliency functions . . . . . . . . . . . 6.1 High Availability and protection in base SONAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.1 Cluster Trivial Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.2 DNS performs IP address resolution and load balancing . . . . . . . . . . . . . . . . . . 6.1.3 File sharing protocol error recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Backup and restore of file data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.1 Tivoli Storage Manager terminology and operational overview . . . . . . . . . . . . . 6.2.2 Methods to backup a SONAS cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.3 TSM client and server considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.4 Configuring interface nodes for TSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.5 Performing TSM backup and restore operations . . . . . . . . . . . . . . . . . . . . . . . . 6.2.6 Using TSM HSM client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.1 Snapshot considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.2 VSS snapshot integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.3 Snapshot creation and management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Local and remote replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.1 Synchronous versus asynchronous replication. . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.2 Block level versus file level replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.3 SONAS cluster replication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.4 Local synchronous replication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.5 Remote async replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Disaster recovery methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.1 Backup of SONAS configuration information . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.2 Restore data from a traditional backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.3 Restore data from a remote replica. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 7. Configuration and sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Tradeoffs between configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.1 Rack configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.2 Switch configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.3 Storage Pod configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.4 Interface Node configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.5 Rack configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Considerations for sizing your configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Inputs for SONAS sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.1 Application characteristics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.2 Workload characteristics definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.3 Workload characteristics impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.4 Workload Characteristics measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Powers of two and powers of ten: the missing space . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Sizing the SONAS appliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5.1 Capacity requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5.2 Storage Subsystem disk type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5.3 Interface node connectivity and memory configuration. . . . . . . . . . . . . . . . . . . . 7.6 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.6.1 Workload analyzer tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chapter 8. Installation planning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 viiiSONAS Architecture and Implementation


7875TOC.fm

8.1 Physical planning considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Installation checklist questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3 Storage considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.1 Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.2 Asynch replication considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.3 Block size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.4 File system overhead and characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.5 SONAS master file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.6 Failure groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.7 Setting up storage pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 SONAS integration into your network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.1 Authentication using AD or LDAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.2 Planning IP addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.3 Data access and IP address balancing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5 Attachment to customer applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.1 Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.2 Share access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.3 Caveats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.4 Backup considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 9. Installation and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1 Pre-Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.1 Hardware installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.2 Software installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.3 Check health of the Node hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.4 Additional hardware health checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3 Post Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 Software Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5 Sample environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.1 Initial hardware installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.2 Initial software configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.3 Understanding the IP Addresses for Internal Networking . . . . . . . . . . . . . . . . . . 9.5.4 Configure the Cluster Manager (CTDB) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.5 List all available Disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.6 Adding a second Failure Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.7 Create the GPFS File System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.8 Configure the DNS Server IP addresses and domains. . . . . . . . . . . . . . . . . . . . 9.5.9 Configure the NAT Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.10 Configure Authentication - AD and LDAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.11 Configure Data Path IP Addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.12 Configure Data Path IP Address Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.13 Attach the Data Path IP Address Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6 Creating Exports for data access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.7 Modify ACLs to the shared export . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.8 Test access to the SONAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 10. SONAS administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1 Using the management interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1.1 GUI tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1.2 Accessing the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2 SONAS administrator tasks list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3 Cluster Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

244 248 254 254 255 256 257 257 258 258 259 259 262 262 271 271 272 272 272 273 274 274 274 274 275 275 276 276 277 277 285 286 286 287 288 288 290 292 294 296 297 297 297 298 301 305 306 306 337 341 343

Contents

ix

7875TOC.fm


10.3.1 Add/Delete cluster to the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.2 View Cluster status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.3 View Interface Node and Storage Node Status: . . . . . . . . . . . . . . . . . . . . . . . . 10.3.4 Modify Interface and Storage Nodes status . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4 File system management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.1 Create a File system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.2 List the Filesystem status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.3 Mount the FIle system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.4 Unmount the File system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.5 Modify the File system configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.6 Delete File system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.7 Master and Non-Master file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.8 Quota Management for File systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.9 Fileset management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5 Creating and managing exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.1 Create Exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.2 List and view status of exports created . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.3 Modify exports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.4 Remove service/protocols. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.5 Activate Exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.6 Deactivate Exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.7 Remove Exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.8 Test accessing the Exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6 Disk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.1 List Disks and View Status:. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.2 Change Properties of disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.3 Start Disks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.4 Remove Disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7 User management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.1 SONAS administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.2 SONAS end users. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8 Services Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.1 Management Service administration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.2 Manage Services on the cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.9 Real-time and historical reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.9.1 System Utilization: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.9.2 File System Utilization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.9.3 Utilization Thresholds and Notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.10 Scheduling tasks in SONAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.10.1 List tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.10.2 Remove task: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.10.3 Modify the Schedule Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11 Health Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.1 Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.2 Default Grid view. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.3 Event logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.12 Call home . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 11. Migration overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 SONAS file system authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.1 SONAS file system ACLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.2 File sharing protocols in SONAS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.3 Windows CIFS and SONAS considerations . . . . . . . . . . . . . . . . . . . . . . . . . . .

343 343 343 344 345 345 352 352 354 355 360 361 361 363 369 369 374 374 377 378 379 380 380 385 385 388 390 390 390 390 394 396 396 397 402 403 405 407 408 408 410 411 412 412 415 423 425 427 428 428 429 430

x



7875TOC.fm

11.2 Migrating files and directories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.1 Data migration considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.2 Metadata migration considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.3 Migration tools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Migration of CIFS Shares and NFS Exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4 Migration considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.1 Migration data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.2 Types of migration approaches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.3 Sample throuhput estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.4 Migration throughput example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 12. Getting started with SONAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1 Quick start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.1 Quick start tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Connecting to the SONAS system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.1 Connect to SONAS appliance using GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.2 Connect to SONAS appliance using CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3 Create SONAS administrators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.1 Creating a SONAS administrator using the CLI . . . . . . . . . . . . . . . . . . . . . . . . 12.3.2 Creating a SONAS administrator using the GUI . . . . . . . . . . . . . . . . . . . . . . . . 12.4 Monitoring your SONAS environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.1 Topology view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.2 SONAS Logs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.3 Performance and reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.4 Threshold monitoring and notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5 Create a filesystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.1 Creating a filesystem using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.2 Creating a filesystem using the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.6 Creating an export. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.6.1 Configuring exports using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.6.2 Configuring exports using the CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.7 Accessing an export . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.7.1 Accessing a CIFS share from Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.7.2 Accessing a CIFS share from Windows command prompt. . . . . . . . . . . . . . . . 12.7.3 Accessing a NFS share from Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.8 Creating and using snapshots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.8.1 Creating snapshots with the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.8.2 Creating snapshots with the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.8.3 Accessing and using snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.9 Backing up and restoring data with TSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 13. Hints, tips and how to information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1 What to do when you receive an error message. . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.1 EFSSG0026I management service stopped. . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2 Debugging SONAS with Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.1 CTDB Health Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.2 GPFS Logs: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.3 CTDB Logs: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.4 Samba/Winbind Logs: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3 CTDB Unhealthy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.1 CTDB manages Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.2 Master file system umounted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.3 CTDB manages GPFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

431 432 433 434 436 436 437 437 439 439 441 442 442 442 442 443 444 444 444 446 446 449 449 451 454 454 457 458 458 461 461 462 462 463 464 464 465 465 466 469 470 470 470 470 471 471 471 471 471 472 472

Contents

xi

7875TOC.fm


13.3.4 GPFS unable to mount . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473 Appendix A. Additional component detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CTDB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . File system concepts and access permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . GPFS overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . GPFS Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . GPFS File Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . GPFS Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . GPFS High Availability solution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . GPFS failure group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Other GPFS features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tivoli Storage Manager (TSM) overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tivoli Storage Manager concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tivoli Storage Manager architectural overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tivoli Storage Manager storage management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Policy management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hierarchical storage management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How to get Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475 476 487 489 492 493 494 497 498 498 499 499 500 506 509 512 517 517 517 517 518

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519

xii



7875spec.fm

NoticesThis information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.

Copyright IBM Corp. 2010. All rights reserved.

xiii

7875spec.fm


TrademarksIBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol ( or ), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:AFS AIX BladeCenter DB2 Domino Enterprise Storage Server eServer FlashCopy GPFS HACMP IBM Lotus PowerVM pSeries Redbooks Redbooks (logo) System i System p5 System Storage System x Tivoli XIV xSeries z/OS

The following terms are trademarks of other companies: Snapshot, and the Network Appliance logo are trademarks or registered trademarks of Network Appliance, Inc. in the U.S. and other countries. Java, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows NT, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.

xiv



7875pref.fm

PrefaceIBM Scale Out Network Attached Storage (SONAS) is a Scale Out NAS offering designed to manage vast repositories of information in enterprise environments requiring very large capacities, high levels of performance, and high availability. The IBM Scale Out Network Attached Storage provides a range of reliable, scaleable storage solutions for a variety of storage requirements. These capabilities are achieved by using network access protocols such as NFS, CIFS, HTTP and FTP. Utilizing built-in RAID technologies all data is well protected with options to add additional protection through mirroring, replication, Snapshots and backup. These storage systems are also characterized by simple management interfaces that make installation, administrating and troubleshooting uncomplicated and straight forward. This book provides the reader with details of the hardware and software architecture that make up the SONAS appliance along with configuration, sizing and performance considerations. It provides information of the integration of the SONAS into an existing network. The administration of the SONAS appliance through the GUI and CLI is demonstrated as well as backup and availability scenarios. A quick start scenario takes you through common SONAS administration tasks to familarize you with the SONAS system.

The team who wrote this bookThis book was produced by a team of specialists from around the world working at the International Technical Support Organization, San Jose Center. Mary Lovelace is a Consulting IT Specialist at the International Technical Support Organization. She has more than 20 years of experience with IBM in large systems, storage, and storage networking product education, system engineering and consultancy, and systems support. She has written many Redbooks publications about IBM Tivoli Storage Productivity Center, Tivoli Storage Manager, and z/OS storage products. Vincent Boucher is an IT Specialist as a member of the EMEA Products and Solutions Support Center (PSSC) of Montpellier France. His role within the Storage Benchmark team is to demonstrate the efficiency of IBM solutions and their added value to customers. He holds a Engineering degree in Mathematics and Computer Science from the ENSEEIHT Engineering schools in Toulouse. Vincents area of expertise include Linux, IBM systems x, mid-range IBM Storage, and GPFS from both his past High Performance Computing and new Storage benchmark experiences. Shradha Nayak is a staff software engineer working with IBM India software Labs in Pune, India. She holds a Bachelor of Computer Science Engineering degree and has around 6.5 years of experience. She has been working in the storage domain since and has good expertise in Scale out File Service (SoFS) and Scale Out Network Attached Storage (SONAS). Prior to this, she worked as a Level-3 developer for Distributed File Service (DFS) and also worked for AFS. Shradha is focussing on storage products and cloud storage and is currently part of the Level-3 developers teams for SONAS. Being a part of the SONAS developing and testing team, she has developed a thorough knowledge of SONAS, its components and working. In this book, she has mainly focussed on the Installation, Configuration and Administration of SONAS. Shradha is also interested in social media and social networking tools and methodologies.


xv

7875pref.fm


Curtis Neal is an Executive IT Specialist working for the System Storage Group in San Jose, California. He has over 25 years of experience in various technical capacities, including mainframe and open system test, design, and implementation. For the past eight years, he has led the Open Storage Competency Center, which helps customers and IBM Business Partners with the planning, demonstration, and integration of IBM System Storage Solutions. Lukasz Razmuk is an IT Architect at IBM Global Technology Services in Warsaw, Poland. He has six years of IBM experience in designing, implementing and supporting solutions in AIX, Linux, pSeries, virtualization, high availability, GPFS, SAN Storage Area Network, Storage for Open Systems and IBM Tivoli Storage Manager. Moreover he acts as a Technical Account Advocate for Polish clients. He holds a master's of science degree in Information Technology from Polish-Japanese Institute of Information Technology in Warsaw and also many technical certifications including IBM Certified Advanced Technical Expert System p5, IBM Certified Technical Expert pSeries HACMP, Virtualization Technical Support and Enterprise Technical Support AIX 5.3. John Sing John Sing is an Executive IT Consultant with IBM Systems and Technology Group. John has specialties in large Scale Out NAS, in IT Strategy and Planning, and in IT High Availability and Business Continuity. Since 2001, John has been an integral member of the IBM Systems and Storage worldwide planning and support organizations. He started in the Storage area in 1994 while on assignment to IBM Hong Kong (S.A.R. of China), and IBM China. In 1998, John joined the Enterprise Storage Server Planning team for PPRC, XRC, and FlashCopy. He has been the marketing manager for these products, and in 2002, began working in Business Continuity and IT Strategy and Planning. Since 2009, John has also added focus on IT Competitive Advantage strategy including Scale Out NAS and Cloud Storage. John is the author of three ITSO Redbooks on these topics, and in 2007, celebrated his 25th anniversary of joining IBM. .John Tarella is a Senior Consulting IT Specialist who works for IBM Global Services in Italy. He has twentyfive years of experience in storage and performance management on mainframeand distributed environments. He holds a degree in Seismic Structural Engineering from Politecnico di Milano, Italy. His areas of expertise include IBM Tivoli Storage Manager and storage infrastructure consulting, design, implementation services, open systems storage, and storage performance monitoring and tuning. He is presently focusing on storage solutions for business continuity, information lifecycle management, and infrastructure simplification. He has written extensively on z/OS DFSMS, IBM Tivoli Storage Manager, SANs, storage business continuity solutions, content management and ILM solutions. John is currently focusing on cloud storage delivery. He also has an interest in Web2.0 and social networking tools and methdologies.

xvi



7875pref.fm

Figure 0-1 The team, from left: Curtis, Lukasz, John Tarella, Mary, Vincent, Shradha, John Sing

Thanks to the following people for their contributions to this project: Sven Oehme Mark Taylor Alexander Saupp Mathias Dietz Jason Auvenshine Greg Kishi Scott Fadden Leonard Degallado Todd Neville Warren Saltzman Wen Moy Tom Beglin Adam Childers Frank Sowin Pratap Banthia Dean Hanson Everett Bennally Ronnie Sahlberg Christian Ambach Andreas Luengen Bernd Baeuml

Preface

xvii

7875pref.fm


Now you can become a published author, too!Here's an opportunity to spotlight your skills, grow your career, and become a published author - all at the same time! Join an ITSO residency project and help write a book in your area of expertise, while honing your experience using leading-edge technologies. Your efforts will help to increase product acceptance and customer satisfaction, as you expand your network of technical contacts and relationships. Residencies run from two to six weeks in length, and you can participate either in person or as a remote resident working from your home base. Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.html

Comments welcomeYour comments are important to us! We want our books to be as helpful as possible. Send us your comments about this book or other IBM Redbooks publications in one of the following ways: Use the online Contact us review Redbooks form found at: ibm.com/redbooks Send your comments in an e-mail to: [email protected] Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400

xviii



7875Overview_Concept.fm

1

Chapter 1.

Introduction to Scale Out File Network Attached StorageSONAS is designed to address the new storage challenges posed by the continuing explosion of data. Leveraging mature technology from IBMs High Performance Computing experience, and based upon IBMs General Parallel File System (GPFS), SONAS is an easy-to-install, turnkey, modular, scale out NAS solution that provides the performance, clustered scalability, high availability and functionality that are essential to meeting strategic Petabyte Age and cloud storage requirements. The high-density, high-performance SONAS can help organizations consolidate and manage data affordably, reduce crowded floor space, and reduce management expense associated with administering an excessive number of disparate storage systems. With its advanced architecture, SONAS virtualizes and consolidates multiple filers into a single, enterprise-wide file system, which can translate into reduced total cost of ownership, reduced capital expenditure and enhanced operational efficiency.


1



1.1 Marketplace requirementsThere are several factors driving the need for a new way of looking at information and the way we make decisions based on that information. When you think of the changes in our world today the instrumentation, interconnectedness and intelligence of our environments this produces a massive glut of new information, from new sources, with new needs to leverage it. This exacerbates some of the challenges that we have been dealing with for awhile now, just on a whole new scale. There is an explosion in the amount of data of course, but also shifts in the nature of data (see Figure 1-1 on page 2). Once virtually all the information available to be "processed" was authored by someone. Now that kind of data is being overwhelmed by machine-generated data spewing out of sensors, RFID, meters, microphones, surveillance systems, GPS systems and all manner of animate and inanimate objects. With this expansion of the sources of information comes large variance in the complexion of the available data -- very noisy, lots of errors -- and no time to cleanse it in a world of real-time decision making. Also, consider that todays economic times require corporations and governments to analyze new information faster and make timely decisions for achieving business goals. As the volume, variety and velocity of information and decision making increases, this places a larger burden on organizations to effectively and efficiently distribute the right information, at the right time, to the people, processes and applications that are reliant upon that information to make better business decisions. All of these situations are creating challenges and it also provides an excellent opportunity for driving an information-led transformation.

Figure 1-1 Explosion of data demands an Information -Led transformation

2




Todays businesses are demanding the ability to create, manage, retrieve, protect and share business and social digital content or large rich media files over a broadband Internet that reaches to every corner of the globe (see Figure 1-2 on page 3 ). Users are creating and using data that is redefining our business and social world in real time. Unlike traditional IT data, this rich digital content is almost entirely file-based or object-based, and it is growing ever larger in size, with highly diverse and unpredictable usage patterns.

Figure 1-2 Todays workloads demand new approach to data access

Innovative applications in business analytics, digital media, medical data and cloud storage are creating requirements for data access rates and response times to individual files that were previously unique to high-performance computing environmentsand all of this is driving a continuing explosion of business data. While many factors are contributing to data growth, these trends are significant contributors: Digital representation of physical systems and processes Capture of digital content from physical systems and sources Deliveries of digital content to a global population Additional trends are driven by the following kinds of applications: Product Life Cycle Management (PLM) systems, which include Product Data Management systems and mechanical, electronic, and software design automation Service Life Cycle Management (SLM) systems Information Life Cycle Management (ILM), including e-mail archiving Video on demand: Online, broadcast, and cable Digital Video Surveillance (DVS): Government and commercial Video animation rendering Seismic modeling and reservoir analysis Pharmaceutical design and drug analysis Digital health care systems

Chapter 1. Introduction to Scale Out File Network Attached Storage

3



Web 2.0 and service-oriented architecture

When it comes to traditional IT workloads, traditional storage will continue to excel for the traditional applications for which they were designed. But solutions like Network Attach Storage (NAS) were not intended to scale to the high levels and extremely challenging workload characteristics required by todays Internet-driven, Petabyte Age applications.

1.2 Understanding I/OA major source of confusion regarding NAS is the concept of File I/O versus Block I/O. Understanding the difference between these two forms of data access is crucial to realizing the potential benefits of any SAN-based or NAS-based solution.

1.2.1 File I/OWhen a partition on a hard drive is under the control of an operating system (OS), the OS will format it. Formatting of the partition occurs when the OS lays a file system structure on the partition. This file system is what enables the OS to keep track of where it stores data. The file system is an addressing scheme the OS uses to map data on the partition. Now, when you want to get to a piece of data on that partition, you must request the data from the OS that controls it. For example, suppose that Windows 2000 formats a partition (or drive) and maps that partition to your system. Every time you request to open data on that partition, your request is processed by Windows 2000. Since there is a file system on the partition, it is accessed via File I/O. Additionally, you cannot request access to just the last 10 KB of a file. You must open the entire file, which is another reason that this method is referred to as File I/O. Using File I/O is like using an accountant. Accountants are good at keeping up with your money for you, but they charge you for that service. For your personal checkbook, you probably want to avoid that cost. On the other hand, for a corporation where many different kinds of requests are made, an accountant is a good idea. That way, checks are not written when they should not be. A file I/O specifies the file. It also indicates an offset into the file (see Figure 1-3 on page 5). For instance, the I/O may specify Go to byte 1000 in the file (as if the file were a set of contiguous bytes), and read the next 256 bytes beginning at that position. Unlike block I/O, there is no awareness of a disk volume or disk sectors in a file I/O request. Inside the NAS appliance, the operating system keeps track of where files are located on disk. It is the NAS OS which issues a block I/O request to the disks to fulfill the client file I/O read and write requests it receives. By default, a database application that is accessing a remote file located on a NAS device is configured to run with File System I/O. It cannot utilize raw I/O to achieve improved performance.

4




Figure 1-3 File I/O

1.2.2 Block I/OBlock I/O (raw disk) is handled differently (see Figure 1-4 on page 5 ). There is no OS format done to lay out a file system on the partition. The addressing scheme that keeps up with where data is stored is provided by the application using the partition. An example of this would be DB2 using its tables to keep track of where data is located rather than letting the OS do that job. That is not to say that DB2 cannot use the OS to keep track of where files are stored. It is just more efficient, for the database to bypass the cost of requesting the OS to do that work.

Figure 1-4 Block I/O

When sharing files across a network, something needs to control when writes can be done. The operating system fills this role. It does not allow multiple writes at the same time, evenChapter 1. Introduction to Scale Out File Network Attached Storage

5



though many write requests are made. Databases are able to control this writing function on their own so in general they run faster by skipping the OS although this depends on the efficiency of the implementation of file system and database.

1.2.3 Network Attached Storage (NAS)Storage systems which optimize the concept of file sharing across the network have come to be known as NAS. NAS solutions utilize the mature Ethernet IP network technology of the LAN. Data is sent to and from NAS devices over the LAN using TCP/IP protocol. One of the key differences in a NAS appliance, compared to direct attached storage (DAS) or other network storage solutions such as SAN or iSCSI, is that all client I/O operations to the NAS use file level I/O protocols. File I/O is a high level type of request that, in essence, specifies only the file to be accessed, but does not directly address the storage device. This is done later by other operating system functions in the remote NAS appliance. By making storage systems LAN addressable, the storage is freed from its direct attachment to a specific server, and any-to-any connectivity is facilitated using the LAN fabric. In principle, any user running any operating system can access files on the remote storage device. This is done by means of a common network access protocolfor example, NFS for UNIX servers and CIFS for Windows servers. In addition, a task such as backup to tape can be performed across the LAN using software like Tivoli Storage Manager (TSM), enabling sharing of expensive hardware resources (for example, automated tape libraries) between multiple servers.

NAS file system access and administrationNetwork access methods like NFS and CIFS can only handle file I/O requests to the remote file system. This is located in the operating system of the NAS device. I/O requests are packaged by the initiator into TCP/IP protocols to move across the IP network. The remote NAS file system converts the request to block I/O and reads or writes the data to the NAS disk storage. To return data to the requesting client application, the NAS appliance software repackages the data in TCP/IP protocols to move it back across the network. A storage device cannot just attach to a LAN. It needs intelligence to manage the transfer and the organization of data on the device. The intelligence is provided by a dedicated server to which the common storage is attached. It is important to understand this concept. NAS comprises a server, an operating system, and storage which is shared across the network by many other servers and clients. So a NAS is a specialized server or appliance, rather than a network infrastructure, and shared storage is attached to the NAS server. However, NAS filers today do not scale to high capacities. When one filer was fully utilized, a second, third, and more filers were installed. The result - administrators found themselves in the managing 'silos' of filers. Capacity on individual filers could not be shared. Some filers were heavily accessed while others were mostly idle. Managing the many different filers adds complexity to the administrator's job. When adding more storage capacity to some filers; we cannot add more performance to a particular file, than what is possible with the single disk drive or controller that the filer typically uses. In other words, there is limited parallelism, (typically, perhaps one, two, or a few controllers) for serving an individual file. Figure 1-5 on page 7 is an summary of traditional NAS limitations.

6




Figure 1-5 Network Attached Storage limitations

This is compounded by the fact that at 100s of TBs or more, conventional backup of such a large storage farm is difficult, if not impossible. There is also the issue that even though one might be using incremental only backup, but *scanning* 100s of TBs to *identify* the changed files or changed blocks could in itself take to long, with too much overhead. More issues include that there may not be any way to centrally and automatically apply file placement, migration, deletion, and management policies from one centrally managed, centrally deployed control point. Doing manual management of tens or 100s of filers was proving to be neither timely nor cost-effective, and effectively prohibited any feasible way to globally implement automated tiered storage.

1.3 Scale Out Network Attached Storage (SONAS)IBM Scale Out Network Attached Storage (SONAS) is designed to address the new storage challenges posed by the continuing explosion of data. IBM recognizes that a critical component of future enterprise storage is a scale-out architecture that takes advantage of industry trends to create a truly efficient and responsive storage environment, eliminating the waste created by the proliferation of scale-up systems and providing a platform for file server consolidationand that is where SONAS comes in (see Figure 1-6 on page 8). Leveraging mature technology from IBMs High Performance Computing experience, and based upon IBMs flagship General Parallel File System (GPFS), SONAS is an easy-to-install, turnkey, modular, scale out NAS solution that provides the performance, clustered scalability, high availability and functionality that are essential to meeting strategic Petabyte Age and cloud storage requirements. Simply put, SONAS is a scale-out storage system combined with high-speed interface nodes interconnected with storage capacity and GPFS, which enables organizations to scale


7



performance alongside capacity in an integrated, highly-available system. The high-density, high-performance SONAS can help your organization consolidate and manage data affordably, reduce crowded floor space, and reduce management expense associated with administering an excessive number of disparate storage systems.

Figure 1-6 IBM SONAS overview

1.3.1 SONAS architectureThe SONAS system is available in as small a configuration as 20 terabytes (TB) usable in the base rack, up to a maximum of 30 interface nodes and 60 storage nodes in 30 storage pods. The storage pods fit into 15 storage expansion racks. The 60 storage nodes can contain a total of 7200 hard-disk drives when fully configured and you are using 96-port InfiniBand switches in the base rack. With its advanced architecture, SONAS virtualizes and consolidates multiple filers into a single, enterprise-wide file system, which can translate into reduced total cost of ownership, reduced capital expenditure, and enhanced operational efficiency. Figure 1-7 on page 9 provides a high level overview of the SONAS architecture.

8




Figure 1-7 SONAS architecture

Assuming 2 TB SATA disk drives, such a system has 14.4 petabytes (PB) of raw storage and billions of files in a single large file system. You can have as few as eight file systems in a fully configured 14.4 PB SONAS system or as many as 256 file systems. It provides an automated policy-based file management that controls backups and restores, snapshots, and remote replication. It also provides: A single global namespace with logical paths that do not change because of physical data movement Support for Serial Attached SCSI (SAS) and Serial Advanced Technology Attachment (SATA) drives High-availability and load-balancing Centralized management Centralized backup An interconnected cluster of file-serving and network-interfacing nodes in a redundant high-speed data network Virtually no capacity limits Virtually no scalability limits IBM Call Home trouble reporting and IBM Tivoli Assist On Site (AOS) remote support capabilities Enhanced support for your Tivoli Storage Manager Server product, with a preinstalled Tivoli Storage Manager client Support for the cloud environment. A controlled set of end users, projects, and applications can: Share files with other users within one or more file spaces Control access to their files using access control lists (Microsoft Windows clients) and user groups


9



Manage each file space with a browser-based tool

Global namespaceSONAS provides a global namespace that enables your storage infrastructure to scale to extreme amounts of data, from terabytes to petabytes. Within the solution, centralized management, provisioning, control, and automated information life-cycle management (ILM) are integrated as standard features to provide the foundation for a truly cloud storage enabled solution.

interface nodesThe high-performance interface nodes provide connectivity to your Internet Protocol (IP) network for file access and support of both 1-gigabit Ethernet (GbE) and 10-GbE connection speeds.Each interface node can connect to the IP network with up to eight separate data-path connections. Performance and bandwidth scalability are achieved by adding interface nodes, up to the maximum of 30 nodes, each of which has access to all files in all file systems You can scale out to thirty interface nodes. Each interface node has its own cache memory, so you increase caching memory and data paths in your file-serving capacity by adding an interface node. Of course, you also increase file-serving processor capacity. If raw storage capacity is the prime constraint in the current system, the SONAS system scales out to as much as 14.4 petabytes (PBs) with 2 terabyte (TB) SATA drives, with up to 256 file systems that can each have up to 256 file-system snapshots. Most systems that a SONAS system typically displaces cannot provide clients with access to so much storage from a single file-serving head. Every interface node has access to all of the storage capacity in the SONAS system.

1.3.2 SONAS scale out capabilitySONAS provides extreme scale out capability, a globally clustered NAS file system built upon IBM GPFS. The global namespace is maintained across the entire global cluster of multiple storage pods and multiple interface nodes. All interface nodes and all storage nodes share equally in the cluster to balance workloads dynamically and provide parallel performance to all users and storage, while also assuring high availability and automated failover. SONAS is a scalable virtual file storage platform that grows as data grows. It meets demanding performance requirements as new processors can be added independently or as storage capacity is added, eliminating a choke point found in traditional scale-up systems. SONAS is designed for high availability 24x7 environments with a clustered architecture that is inherently available and, when combined with the global namespace, allows for much higher utilization rates than found in scale-up environments.

1.3.3 SONAS softwareSONAS software leverages a powerful cross-platform access to the same files with locking for data integrity. In addition, SONAS provides high availability Linux, UNIX, CIFS (Windows) sessions with no client side changes (see Figure 1-8 on page 11). Deploying SONAS allows users to reduce the overall number of disk drives and file storage systems that need to be housed, powered, cooled, and managed relative to scale-up systems.

10




Figure 1-8 SONAS software

Storage Management benefitsSONAS also provides integrated support of policy-based automated placement and subsequent tiering and migration of data. Customers can provision storage pools and store file data according to its importance to the organization. For example, a user can define multiple storage pools with different drive types and performance profiles. They can create a higher performance storage pool with SAS drives and define a less expensive (and lower performance) pool with SATA drives. Rich, sophisticated policies are built into SONAS which can transparently migrate data between pools based on many characteristics, such as capacity threshold limits and age of the data. This helps to address business critical performance requirements. Leveraging automated storage tiering, users can finally realize the cost savings and business benefits of information lifecycle management (ILM) at an immense scale.

1.3.4 High availability designSONAS provides a NAS storage platform for global access of your business critical data. Your business critical data can be secured with both information protection and business continuity solutions, giving you a high level of business continuity assurance. In the event of data corruption or an unexpected disaster that could harm corporate data, SONAS helps you to recover and quickly resume normal enterprise and data center operations (see Figure 1-9 on page 12). SONAS supports large enterprise requirements for remote replication, point-in-time copy (file system-level snapshots), and scalable automated storage tiering, all managed as a single instance within a global namespace. SONAS asynchronous replication is specifically designed to cope with connections that provide low bandwidth, high latency and low


11



reliability. The async scheduled process will pick-up the updates on the source SONAS system and write them to the target SONAS system, which may be thousands of miles away. Security and information protection are enhanced in a consolidated SONAS environment. For example, users considering the implementation of security and protection solutions are concerned about maintaining data availability as systems scalea key design point for SONAS. Its clustered architecture is designed for high availability at scale and 24 x 7 x Forever operation, complementing consolidated security and protection solutions to provide an always-on information infrastructure.

Figure 1-9 High availability and disaster recovery design

1.4 SONAS architectural concepts and principlesIn this section we review the overall SONAS architecture and operational principles. We will start with the logical diagram in Figure 1-10.

12




Logical Diagram of IBM SONAS/home /app l /data /web

Logical

/home/appl/data/web/important_big_spreadsheet.xls /home/appl/data/web/important_big_spreadsheet.xls /home/appl/data/web/big_architecture_drawing.ppt /ho me/appl/data/web/big_architecture_drawing.ppt /home/appl/data/web/unstructured_big_video.mpg /ho me/appl/data/web/unstructured_big_video.mpg

IBM Scale Out NAS

Physical

Global NamespacePolicy EngineInte rfa c e node s

Inter fac e nodes

..

Interfa ce node s

> scale out ... > scaleoutStorage Pool (Tier 3) with 2TB SATA

Stor age node s

..

Stor age node s

St ora ge nodes

Storage Pool (Tier 1): SAS drives

Storage Pool (Tier 2) with 1TB SATA

Figure 1-10 Logical Diagram of IBM SONAS

In the top half of this diagram, we see the logical file directory structure as seen by the users. SONAS presents and preserves this same logical appearance to the users, no matter what we do to physically manage these files, and all files in the SONAS, from creation to deletion. The user sees only his global namespace, his user directories and files. As a SONAS expands, manages, and changes the physical data location and supporting physical infrastructure, the users will still have the unchanged appearance of one single logical global namespace, and maintain their logical file structure without change.. In the lower half of this diagram, we see a representation of the SONAS internal architectural components. SONAS has interface nodes, which serve data to and from the users, over the network. SONAS also has storage nodes, which service the storage for the SONAS clustered file system. All SONAS nodes are in a global cluster, connected via Infiniband. All interface nodes have full read/write access to all storage nodes. All storage nodes have full read/write access to all interface nodes. Each of the nodes runs a copy of IBM SONAS Software (5639-SN1), which provides all the functions of SONAS, including a Cluster Manager which manages the cluster and dispatches workload evenly across the cluster. Also included is the SONAS central storage policy engine, which runs in a distributed fashion across all the nodes in the SONAS. The SONAS policy engine provides central management of the lifecyle of all files, in a centrally deployed, centrally controlled, enforceable manner. The policy engine function is not tied to a particular node, it executes in a distributed manner across all nodes. Not shown is the SONAS management node(s), which monitors the health of the SONAS. IBM SONAS Software manages the cluster and maintains the coherency and consistency of the file system(s), providing file level and byte level locking, using a sophisticated distributed, token (lock) management architecture that is derived from IBM General Parallel File System


13



(GPFS) technology. As we shall see, the SONAS clustered grid architecture provides the foundation for automatic load balancing, high availability, scale out high performane, with multiple paralle, concurrent writers and readers. Physical disk drives are allocated to SONAS logical storage pools. Typically, we would allocate a high performance pool of storage (which uses the fastest disk drives), and a lower tier of storage for capacity (less expensive, slower spinning drives). In the example above, we have allocated three logical storage pools.

1.4.1 Create, write, and read filesTo understand the operation of SONAS Software and the interaction of the SONAS Software functions, it is best to follow the lifecycle of a file as it flows from creation through automated tiered storage management, to eventual destaging to backups, external storage, or deletion. We will follow the lifecycle of reading and writing files, as they traverse the SONAS Software and the SONAS central policy engine, and in this way, well see how the SONAS software and policy engine are used to manage these files in three different logical storage pools. Via easy to write rules, a SONAS administrator may automate the storage management , file placement, and file migration within a SONAS. The central policy engine in SONAS is the one central location to provide a enforceable powerful set of policies and rules to globally manage all physical storage, while preserving the appearance of a global namespace and an unchanging file system to the users.

1.4.2 Creating and writing a fileWhen a create file request comes into SONA

SONAS Architecture and Implementation

Documents