Fibre Channel Basics

Cisco Systems CAE.

Fibre Channel Basics

Cisco Systems CAE.

SCSI I/O Channel

Small Computer System Interface - SCSI

  SCSI is a standard that defines an interface between an initiator (usually a computer) and a target (usually a storage device such as a hard disk)

  Logical Unit Number (LUN): A 64-bit field within SCSI that identifies the Logically addressable Unit within a target SCSI device

Host (Initiator) Disk (Target)

READ DATA DATA DATA

WRITE DATA DATA DATA

STATUS

STATUS

SCSI I/O Channel STATUS STATUS

Disk (Target) Host (Initiator)

SCSI READ OPERATION

SCSI WRITE OPERATION READY

Cisco Systems CAE.

The SCSI I/O Channel SCSI I/O Channel provides half-

duplex pipe for SCSI command, data, and status

SCSI I/O channel can be internal or external to host

Multiple SCSI I/O channels can exist within host

A network approach can scale the I/O channel in many areas (length, devices, throughput)

NIC Adapter SCSI Adapter

NIC Driver

TCP/IP Stack

Adapter Driver

SCSI Generic

Block Device

File System

Applications Half-Duplex I/O Channel

SCSI

SCSI Initiator

SCSI Target

Cisco Systems CAE.

Fibre Channel Fabric

SCSI

Target

Fibre Channel HBA

Host System

Initiator

Fibre Channel I/O Networking Fibre Channel provides high speed transport

for SCSI payload via Host Buss Adapter (HBA)

Fibre Channel overcomes many shortcomings of Parallel I/O including: Addressing for up to 16 million nodes Loop (shared) and fabric (switched) transport Host speeds of 100 to 400 MBps (1–4 Gbps) Segments of up to 10km (without extenders) Support for multiple protocols

Combines best attributes of a channel and a network together

Cisco Systems CAE.

Consolidated I/O Networking Traditional Ethernet with Fibre

Channel over Ethernet Consolidated Network Adapter

– (CAN) Native Ethernet NIC Native FC HBA

Relies on Lossless Ethernet PAUSE per IEEE 802.1p

Able to build Ethernet-based SANs using FCoE arrays

FCoE Host

(Initiator)

FCoE (Target)

Storage (Target)

Ethernet Network

FC Fabric FC HBA

Attached Host (Initiator)

FCoE

FCoE

CNA

SCSI

Cisco Systems CAE.

Fibre Channel Communications Point-to-point oriented   Facilitated through device login

N_Port-to-N_Port connection   Logical node connection point

Flow controlled   Buffer-to-buffer credits and end-to-end basis

Acknowledged   For certain classes of traffic, none for others

Multiple connections allowed per device

N_Port Receiver

N_Port

Node Transmitter

Receiver

Link

Transmitter

Node

Cisco Systems CAE.

Fibre Channel Port Types ‘N’ port: Node ports used to connect devices to

switched fabric or point to point configurations ‘F’ port: Fabric ports residing on switches

connecting ‘N’ port devices ‘L’ port: Loop ports are used in arbitrated loop

configurations to build networks without FC switches; these ports often also have ‘N’ port capabilities and are called ‘NL’ ports

‘E’ port: Expansion ports are essentially trunk ports used to connect two Fibre Channel switches

‘GL’ port: A generic port capable of operating as either an ‘E’ or ‘F’ port; it’s also capable of acting in an ‘L’ port capacity; Auto Discovery

N N

N F

NL FL

L L

E E

Cisco Systems CAE.

FL_Port

G_Port

G_Port

G_Port

E_Port

F_Port

F_Port

F_Port

E_Port

Node N_Port

Node N_Port

Node N_Port

Node NL_Port

Node NL_Port

Node NL_Port

Fibre Channel Port Types Fibre Channel Switch

Fabric Switch

Input Port

Output Port

Fabric X

Cisco Systems CAE.

Inter-Switch Link (ISL)

The interconnection between switches is called the ISL   E_Port to E_Port (‘Expansion port)

Supports all classes of service   Class 1, 2, 3, and a special Class F (switch-to-switch)

FC-PH permits consecutive frames of a sequence to be routed over different ISL links for maximum throughput

Cisco’s implementation is to dedicate an FC_ID pair and/or a given exchange to an ISL bundle member to guarantee in-order delivery for exchange/sequence frames

Cisco Extended ISL (EISL, TE port)

EISL

Cisco Systems CAE.

Worldwide Names

Each switch element is assigned a WWN at time of manufacture

Each switch port is assigned a WWN at the time of manufacture

During FLOGI the switch identifies the WWN in the service parameters of the accept frame and assigns a Fibre Channel ID (FCID)

These address assignments can then correlate each fabric port with the switch element

Cisco Systems CAE.

8 Bits

00 00 Arbitrated Loop Physical Address (AL_PA)

Private Loop Device Address Model

Switch Topology Model

Switch Domain Area Device

8 Bits 8 Bits

Public Loop Device Address Model

Switch Domain Area

Arbitrated Loop Physical Address (AL_PA)

Fabric Addressing FC_ID Address Model

FC_ID address models help speed up routing Switches assign FC_ID addresses to N_Ports Some addresses are reserved for fabric services Private loop devices only understand 8-bit address (0x0000xx) FL_Port can provide proxy service for public address translation Maximum switch domains = 239 (based on standard)

Cisco Systems CAE.

Fabric Routing: FSPF

FSPF routes traffic based on destination domain ID For FSPF a domain ID identifies a single switch

This limits the max number of switches that can support in the Fabric to 239 when FSPF is supported

FSPF performs hop-by-hop routing FSPF uses total cost as the metric to determine most

efficient path FSPF supports hierarchical path selection

Provides the scalable routing tables in large topologies

Static routes can be applied

Cisco Systems CAE.

16 Km

1 Gbps FC

4 Gbps FC ~ ½ km per Frame

~ 2 km per Frame

Buffer to Buffer Credit Flow Control BB_Credits and Distance

BB_Credits are used to ensure enough FC frames in flight A full (2112 byte) FC frame is approx 2 km long at 1 Gbps, 1 km long at 2 Gbps and ½ km long at 4 Gbps As distance increases, the number of available BB_Credits need to increase as well Insufficient BB_Credits will throttle performance—no data will be transmitted until R_RDY is returned

2 Gbps FC ~ 1 km per Frame

Cisco Systems CAE.

Fibre Channel Fabric Zoning

Zoning operation Zone members “see” only other members of the zone Zones can be configured dynamically based on WWN Devices can be members of more than one zone FC-AL zoning allows the creation of private loops on a single hub Switched fabric zoning can take place at the port or device level Based on physical switch port Based on device WWN Based on LUN ID

Benefits Secured device access Allows operating system co-existence

SAN

Disk1

Host2 Disk4

Host1

Disk2 Disk3

ZoneA

ZoneB

ZoneC

Cisco Systems CAE.

Virtual SANs (VSANs)

Analogous to VLANs in Ethernet Virtual fabrics created from larger cost-

effective redundant physical fabric Reduces wasted ports of island approach Fabric events are isolated per VSAN—

maintains isolation for HA (i.e. RSCNs) Hardware-based isolation—traffic is explicitly

tagged across interswitch links with VSAN membership info

Statistics can be gathered per VSAN

Physical SAN Islands Are Virtualized onto Common SAN Infrastructure

Cisco MDS 9000 Family with VSAN Service A Virtual SAN (VSAN) Provides a

Method to Allocate Ports within a Physical Fabric to Create Virtual Fabrics

Cisco Systems CAE.