Http:// // The Storage Resource Broker and.

http://www.ngs.ac.ukhttp://www.grid-support.ac.uk

http://www.eu-egee.org/http://www.pparc.ac.uk/http://www.nesc.ac.uk/

The Storage Resource Brokerand the NGS

Slides from Wayne Schroeder, SDSC andPeter Berrisford, RAL

Induction to Grid Computing and the NGS 3

Acknowledgements

• This tutorial selects slides from several sources, specifically from talks given by Wayne Schroeder (SDSC) and Peter Berrisford (RAL)

Induction to Grid Computing and the NGS 4

Goal

• Introduce use of the SRB for distributed file management on the NGS– This is the focus of the practical that follows

• NOTE:– Wayne Schroeder: “SRB does so much, people tend to

learn subsets and are often unaware of useful features”

– So explore further!• http://www.sdsc.edu/srb/

• For a full SRB tutorial, see: http://www.niees.ac.uk/events/srb

Induction to Grid Computing and the NGS 8

SRB Projects• Digital Libraries

– UCB, Umich, UCSB, Stanford,CDL– NSF NSDL - UCAR / DLESE

• NASA Information Power Grid• Astronomy

– National Virtual Observatory – 2MASS Project (2 Micron All Sky Survey)

• Particle Physics – Particle Physics Data Grid (DOE)– GriPhyN – SLAC Synchrotron Data Repository

• Medicine– Digital Embryo (NLM)

• Earth Systems Sciences– ESIPS– LTER

• Persistent Archives– NARA– LOC

• Neuro Science & Molecular Science– TeleScience/NCMIR, BIRN– SLAC, AfCS, …

Over 90 Tera Bytes in 16 million files

Induction to Grid Computing and the NGS 10

What is SRB?

• Storage Resource Broker (SRB) is a software product developed by the San Diego Supercomputing Centre (SDSC).

• Allows users to access files and database objects across a distributed environment.

• Actual physical location and way the data is stored is abstracted from the user

• Allows the user to add user defined metadata describing the scientific content of the information

Induction to Grid Computing and the NGS 11

How SRB Works

MCATDatabase

MCATServer

SRB AServer

SRB BServer

SRBClient

a

b

c d

e

f

g

• 4 major components:

– The Metadata Catalogue (MCAT)

– The MCAT-EnabledSRB Server

– The SRB Storage Server

– The SRB Client

Induction to Grid Computing and the NGS 12

SRB Client Tools

• Provide a user interface to send requests to the SRB server.

• 4 main interfaces: – Command line (S-Commands)

– MS Windows (InQ)

– Web based (MySRB).

– Java (JARGON)

• Web Services (MATRIX)

Induction to Grid Computing and the NGS 13

Planned Deployment on NGS

Online Replication

SRB Server

SRB server @ RAL

Resource Driver

MCAT

SRB server @ Oxford

Resource Driver

Database Servers @ RAL

User

DiskFarm

Disk Farm

MCAT Server @ RAL

DB1

DB n

Database Servers @ Manchester

DB1

DB n

SRB Server

MCAT

MCAT Server @ Manchester

Failover link

DiskFarm

SRB server @ HPCX

Resource Driver

DiskFarm

SRB server @ Leeds

Resource Driver

DiskFarm

Induction to Grid Computing and the NGS 14

Summary

• SRB provides NGS users with– a virtual filesystem

– Accessible from all core nodes and from the “UI” / desktop

– (will provide) redundancy – mirrored catalogue server

– Replica files

– Support for application metadata associated with files

Induction to Grid Computing and the NGS 15

Tutorial Overview

• Use of the Scommands– Commands for unix based access to srb

– Strong analogy to unix file commands

• Accessing files from multiple (two) sites

Induction to Grid Computing and the NGS 16

Getting Started

• Launch two “putty” connections to pub-234– One for commands run on pub-234

– One for connecting to grid-data.rl.ac.uk and running commands from there

• Open browser and follow link from http://homepages.nesc.ac.uk/~gcw/NGS/

• Follow the instructions from there.• Your SRB name is the same as your account on

pub-234