Top Banner

Click here to load reader

Lustre How To

Apr 02, 2015

ReportDownload

Documents

chebrian88

Lustre: A How-To Guide for Installing and Configuring Lustre 1.4.1Date: May 2005

Prepared by Richard Alexander1, Chad Kerner2, Jeffery Kuehn,1 Jeff Layton3 Patrice Lucas4, Hong Ong1 Sarp Oral1 Lex Stein5, Joshua Schroeder6, Steve Woods7, Scott Studham*1

Report No. R05-123562

Prepared at Oak Ridge National Laboratory, managed by UT-Battelle, LLC, for the U.S. Department of Energy under Contract DE-AC05-00OR22725

* To whom correspondence should be sent: [email protected] 1 Oak Ridge National Laboratory (ORNL) 2 National Center for Supercomputing Applications (NCSA) 3 Linux Networx 4 Commissariat lEnergie Atomique (CEA), France 5 Harvard University 6 Chevron 7 MCNC

CONTENTSAbbreviations.................................................................................................................................. v Glossary ........................................................................................................................................ vii Lustre Commands ........................................................................................................................ viii Introduction..................................................................................................................................... 1 Who Should Use this Manual? ................................................................................................... 1 Lustre Version Used in this Document....................................................................................... 1 Other Sources of Lustre Information .......................................................................................... 2 About the Authors....................................................................................................................... 2 Obtaining the Lustre Software ........................................................................................................ 2 Dependencies .............................................................................................................................. 2 Downloading Packages............................................................................................................... 3 Installing Lustre .............................................................................................................................. 3 Prepatched Kernel RPM with Matching Lustre-utils RPM ........................................................ 3 Instructions for Installation ..................................................................................................... 3 Quick Configuration of Lustre................................................................................................ 3 Using the Lustre Wizard for Configuration ................................................................................ 4 Basic Lustre Configuration ............................................................................................................. 6 Setting Up Various Configurations................................................................................................. 7 Single Node Client, MDS, and Two OSTs ................................................................................. 7 Multiple Nodes.......................................................................................................................... 11 Multiple Machines Both OSTs and Clients ....................................................................... 11 Adding OSTs to Existing File Systems ................................................................................ 19 Adding an OST on a New OSS............................................................................................. 19 Adding an OST to an Existing OSS...................................................................................... 23 Shutting Down Lustre ........................................................................................................... 26 Starting Lustre....................................................................................................................... 27 Performance Considerations ......................................................................................................... 27 Estimating Hardware Performance ........................................................................................... 27 Single OST............................................................................................................................ 28 Multiple OSTs....................................................................................................................... 28 Application Performance .......................................................................................................... 28 Testing Tools ............................................................................................................................ 28 Monitoring and Administering with Lustre Manager................................................................... 30 Installation and Configuration .................................................................................................. 31 Lustre Manager Collector ......................................................................................................... 31 Connecting to the Lustre Manager........................................................................................ 32 Importing the Lustre Configuration into the Lustre Manager .............................................. 32 Best Practices ................................................................................................................................ 40 Naming Conventions ................................................................................................................ 40 XML/Shell Configuration File Management............................................................................ 40 MDS/OSS/Client Configuration ............................................................................................... 41 Publish Your Experiences......................................................................................................... 41 LDAP Support .......................................................................................................................... 41

iii

Liblustre Library ....................................................................................................................... 41 Administrative Tips .................................................................................................................. 41 Log Files ............................................................................................................................... 41 Useful Commands................................................................................................................. 41 Appendix A: Sample local.sh file .......................................................................................... 44 Appendix B: Sample XML File.................................................................................................... 48 Appendix C: Building the Lustre Kernel from Source ................................................................. 50 Building Lustre from CFS-Supplied Prepatched Kernel .......................................................... 50 Patching SuSE or Red Hat Kernel Source and Building Lustre ............................................... 52 Building Lustre from Vanilla Sources ...................................................................................... 53

iv

ABBREVIATIONSCFS CVS dcache GNU GPL ID I/O LDAP LMC LOV LUG MDS MSCP NAL NFS nid OBD OSS OST RPM SAN SCA SNMP TCP VFS Cluster File System, Inc. Concurrent Version System Directory Cache GNU is Not Unix General Public License Identification input/output Lightweight Directory Access Protocol Lustre Configuration Maker Logical Object Volume Lustre Users Group Metadata Server Mass Storage Communications Protocol Network Abstraction Layer Network File System Network ID Object Based Devices Object Storage Servers Object Storage Target Red Hat Package Manager Storage Area Network System Communication Architecture Simple Network Management Protocol Transmission Control Protocol Virtual File System

v

GLOSSARYLOV MDS A Logical Object Volume (LOV) is a collection of OSTs into a single volume. The Metadata Server (MDS) maintains a transactional record of high-level file and file system changes. The MDS supports all file system namespace operations, such as file lookups, file creation, and file and directory attribute manipulation. It does not contain any file data, instead redirecting actual file I/O requests to OSTs. The Network Abstraction Layer (NAL) provides out-of-the-box support for multiple types of networks. This layer makes it easy to integrate new network technologies. An Object Storage Server (OSS) is a server node that runs the Lustre software stack. It has one or more network interfaces and usually one or more disks. An Object Storage Target (OST) is a software interface to a single exported backend volume. It is conceptually similar to an network file system (NFS) export, except that an OST does not contain a whole namespace, but rather file system objects.

NAL

OSS

OST

vii

LUSTRE COMMANDSlconf Lustre file system configuration utility This utility configures a node following directives in the . There is a single configuration file for all the nodes in a single cluster. This file should be distributed to all the nodes in the cluster or kept in a location accessible to all the nodes. One option is to store the cluster configuration information in lightweight directory access protocol (LDAP) format on an LDAP server that can be reached from all of the cluster nodes. Low level

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.