IP –Based SAN extensions and Performance Thao Pham CS 622 Fall 07.

IP –Based SAN extensionsIP –Based SAN extensionsand Performanceand Performance

Thao PhamThao Pham

CS 622CS 622

Fall 07Fall 07

Project OverviewProject Overview

To understand main technologies developed To understand main technologies developed to address long distant replication and backup to address long distant replication and backup needs, iSCSI, iFCP and FCIP. needs, iSCSI, iFCP and FCIP.

Ways to optimize performance and maximize Ways to optimize performance and maximize business return on investment (ROI) when business return on investment (ROI) when deploying these technologies.deploying these technologies.

SCSI Transport ProtocolsSCSI Transport Protocols

iSCSIiSCSI

iSCSI facilitates block-level initiator-target iSCSI facilitates block-level initiator-target communication over TCP/IP networks. Thus, communication over TCP/IP networks. Thus, iSCSI completes the storage over IP model, iSCSI completes the storage over IP model, which supported only file-level protocols which supported only file-level protocols (such as Network File System [NFS], (such as Network File System [NFS], Common Internet File System [CIFS], and File Common Internet File System [CIFS], and File Transfer Protocol [FTP]).Transfer Protocol [FTP]).

iSCSIiSCSI

PDUsPDUs

iSCSI Sessions and ConnectionsiSCSI Sessions and Connections Sessions are classified in normal or discovery. iSCSI Sessions are classified in normal or discovery. iSCSI

commands are in normal sessions.commands are in normal sessions. discovery session is used exclusively for iSCSI target discovery session is used exclusively for iSCSI target

discoverydiscovery

iSCSI deploymentiSCSI deployment

iFCPiFCP

iFCP is a gateway-to-gateway protocol, which iFCP is a gateway-to-gateway protocol, which provides Fibre Channel fabric services to Fibre provides Fibre Channel fabric services to Fibre Channel devices over a TCP/IP network. iFCP Channel devices over a TCP/IP network. iFCP uses TCP to provide congestion control, error uses TCP to provide congestion control, error detection and recovery. iFCP's primary detection and recovery. iFCP's primary objective is to allow interconnection and objective is to allow interconnection and networking of existing Fibre Channel devices networking of existing Fibre Channel devices at wire speeds over an IP network. at wire speeds over an IP network.

iFCP deploymentiFCP deployment

iFCP Connection FlowiFCP Connection Flow Following data-link layer initialization, IP initialization occurs.Following data-link layer initialization, IP initialization occurs. iFCP gateways discover each other via iSNS. iFCP gateways discover each other via iSNS. Configuration parameters for the iFCP fabric are discovered via Configuration parameters for the iFCP fabric are discovered via

iSNS.iSNS. IPsec connection is optionally established between each pair of IPsec connection is optionally established between each pair of

iFCP gateways once the iFCP fabric parameters are knowniFCP gateways once the iFCP fabric parameters are known The iSNS then propagates the information to each of the other The iSNS then propagates the information to each of the other

iFCP gateways.iFCP gateways. each iFCP gateway updates the FCNS of its attached FC-SAN each iFCP gateway updates the FCNS of its attached FC-SAN

with the remote node information and creates an entry in its with the remote node information and creates an entry in its address translation table for the remote node. address translation table for the remote node.

iFCP fabric is ready for initiator-target communication. iFCP fabric is ready for initiator-target communication. TCP connection proactively or on request of PLOGITCP connection proactively or on request of PLOGI

FCIPFCIP

Fibre Channel Over TCP/IP (FCIP) describes Fibre Channel Over TCP/IP (FCIP) describes mechanisms that allow the interconnection of mechanisms that allow the interconnection of islands of Fibre Channel storage area networks islands of Fibre Channel storage area networks over IP-based networks to form a unified over IP-based networks to form a unified storage area network in a single Fibre Channel storage area network in a single Fibre Channel fabric. fabric.

FCIP deployment conceptFCIP deployment concept

FCIP tunneling FlowFCIP tunneling Flow IP initialization occurs. IP initialization occurs. The FCIP tunnel parameters configured or discovered The FCIP tunnel parameters configured or discovered

(via SLPv2) IPsec connection (optional) follows.(via SLPv2) IPsec connection (optional) follows. TCP connection establishment. TCP connection establishment. The tunnel initiator transmits an FC Special Frame The tunnel initiator transmits an FC Special Frame

(FSF). FC identifier, identifier of the tunnel initiator, (FSF). FC identifier, identifier of the tunnel initiator, target, and a 64-bit randomly selected number that target, and a 64-bit randomly selected number that uniquely identifies the FSF. uniquely identifies the FSF.

The receiver verifies FSF match its local The receiver verifies FSF match its local configuration. FSF is echoed back to the tunnel configuration. FSF is echoed back to the tunnel initiator. initiator.

After the tunnel initiator receives and verifies the FSF, After the tunnel initiator receives and verifies the FSF, the FCIP tunnel may carry FC traffic.the FCIP tunnel may carry FC traffic.

HP’s XP and EVA storage FCIPHP’s XP and EVA storage FCIPusing HP Continuous Access using HP Continuous Access

End devices include hosts or target storage devices, and fabric services include routing, device discovery, management, authentication, and inter-switch communication.

Performance based on distancePerformance based on distance

Performance function on packet Performance function on packet loss (at 1500km)loss (at 1500km)

iFCP and iSCSI PerformanceiFCP and iSCSI Performancebased on bandwidthbased on bandwidth

iFCPiSCSI

Performance Parameters TuningPerformance Parameters Tuning Increasing TCP window size (depending on round trip Increasing TCP window size (depending on round trip

delay (distance) and bandwidth.delay (distance) and bandwidth. Use of Jumbo Frames (bigger than 1514 bytes up to 9000 Use of Jumbo Frames (bigger than 1514 bytes up to 9000

bytes) -> reduce frame rate.bytes) -> reduce frame rate. Increasing FC buffer creditIncreasing FC buffer credit -> increasing number of frames -> increasing number of frames

to be sent before stopping to wait.to be sent before stopping to wait. Zero copy: data is written to application directly, Zero copy: data is written to application directly,

bypassing memory copy as in TCP (iSCSI does not yet bypassing memory copy as in TCP (iSCSI does not yet support this).support this).

Parallel TCP sessions: enables more efficient use of Parallel TCP sessions: enables more efficient use of available bandwidth, not supported in FCIP.available bandwidth, not supported in FCIP.

Choosing asynchronous vs. synchronous replication.Choosing asynchronous vs. synchronous replication. Distance consideration -> reduce latency.Distance consideration -> reduce latency.

Increase in Window sizeIncrease in Window size

iSCSIiFCP

Parallel TCP sessionsParallel TCP sessionsiFCP iSCSI

ConclusionConclusion

Depending on cost and size of companies, Depending on cost and size of companies, iSCSI are relatively cheaper for companies iSCSI are relatively cheaper for companies without a SAN. Performance can be without a SAN. Performance can be improved by network parameters and TOE improved by network parameters and TOE technologies.technologies.

iFCP and FCIP have high performance iFCP and FCIP have high performance throughput, but they are more costly, throughput, but they are more costly, require hardware implementation and SAN require hardware implementation and SAN infrastructure already in place.infrastructure already in place.

References:References:

Storage Area Network Extension Solutions Storage Area Network Extension Solutions and Performance Assessement by Radha and Performance Assessement by Radha Telikepalli, Tadeusz Drwiega, and James Telikepalli, Tadeusz Drwiega, and James YangYang

Overview of Modern SCSI Networking Overview of Modern SCSI Networking Protocols , by James LongProtocols , by James Long

IETF IETF http://tools.ietf.org/html/rfc3821http://tools.ietf.org/html/rfc3821 HP Continuous Access XP and EVAHP Continuous Access XP and EVA HP SR2122-2HP SR2122-2