3PAR Red Hat Enterprise Linux およびOracle Linux 実装ガ …€¦ · · 2014-04-031概要この実装ガイドでは、Red Hat Enterprise Linux (RHEL) 4、RHEL 5、RHEL 6、および

HP 3PAR Red Hat Enterprise Linux Oracle Linux

Red Hat Enterprise Linux (RHEL) 4RHEL 5RHEL 6 Oracle Linux (OL) HP 3PARStoreServ Storage HP 3PAR StoreServ Storage RHEL

HP : QL226-971402013 10

Copyright 2013 Hewlett-Packard Development Company, L.P.

Hewlett-Packard Company FAR 12.211 12.212 (Commercial ComputerSoftware, Computer Software Documentation, and Technical Data for Commercial Items)

HP HP

Java Oracle Oracle

Red Hat Red Hat Enterprise Linux Red Hat, Inc

UNIX The Open Group

Windows Microsoft Corporation

1 .........................................................................................................6..........................................................................................................6HP 3PAR OS ..........................................................7.................................................................................................................................7

2 HP 3PAR StoreServ Storage ..........................8HP 3PAR OS 3.1.x 2.3.x HP 3PAR StoreServ Storage ........................8

HP 3PAR StoreServ Storage ...........................................8HP 3PAR StoreServ Storage ..............................9.............................................................................................................10

HP 3PAR OS 2.2.x HP 3PAR StoreServ Storage ........................................11................................................................................................11...................................................................................11.............................................................................................................12

HP 3PAR StoreServ Storage ..........................................................................12...........................................................................................12

HP 3PAR ................................................................................................................13 ..................................................................14..........................................................................................15HP 3PAR Priority Optimization.............................................................................................15.......................................................................................................................15.................................................16..................................................................................................18..................................................................................................18

FCoE-to-FC .....................................................................................................................183 HP 3PAR StoreServ Storage iSCSI .............................................20

iSCSI ...................................................................................................20 iSCSI .....................................................................................21RHEL iscsiadm ................................................................................24..............................................................................................25HP 3PAR Priority Optimization..................................................................................................25

4 FCoE HP 3PAR StoreServ Storage ..........................................27FCoE FCoE FCoE .............27..............................................................................................28HP 3PAR Priority Optimization..................................................................................................29

5 ......................................................30...........................................................................30Emulex HBA .........................................................................................................30

Emulex ...............................................................................................30/etc/modprobe.conf ramdisk .....................................................32Emulex HBA NVRAM BIOS .......................................................35

SAN ......................................................35..................................................................................................35

HBACMD Emulex HBA .....................................................36QLogic HBA .........................................................................................................36

QLogic ...............................................................................................37/etc/modprobe.conf ramdisk .....................................................37QLogic HBA NVRAM BIOS .......................................................39SCLI QLogic HBA ............................................................39

Brocade HBA ........................................................................................................40

3

Brocade .............................................................................................40Brocade HBA NVRAM BIOS .....................................................41

Brocade BIOS NVRAM ...............................................42SAN ......................................................42..................................................................................................42BCU Brocade HBA .....................................................42

SCSI .......................................................................................................43UDEV SCSI ..............................................................43

SCSI .......................................................................................44QLogic SCSI ......................................................45Emulex SCSI ......................................................46

..............................................................................................46...................................................................................................47

/etc/multipath.conf ...............................................................................48.....................................................................................................50

Veritas DMP ...........................................................................................51HP 3PAR Host Explorer ...............................................................52

6 iSCSI ...........................................................................53iSCSI iSCSI .........................53RHEL 4 iSCSI .......................................................................................................53

RHEL 4 iSCSI .....................................................................................53RHEL 4 iSCSI ..................................................................53 RHEL 4 iSCSI ..............................................54

RHEL 5 RHEL 6 iSCSI .............................56RHEL 5 RHEL 6 iSCSI ................................................................56RHEL 56 iSCSI .............................................................56RHEL 5 RHEL 6 iSCSI ............................................58

BIOS IP ...............................................................................58OneCommand Manager GUI ..............................................................................61hbacmd .....................................................................................67

RHEL 5 RHEL 6 iSCSI ....................69RHEL 5 RHEL 6 iSCSI ................................................................72RHEL 5 RHEL 6 iscsiadm iSCSI .......72

iSCSI CHAP ............................................................................................74HP 3PAR StoreServ Storage CHAP ..................................................74 RHEL 5 RHEL 6 CHAP .........................................75RHEL 4 CHAP ...................................................................................76

HP 3PAR StoreServ Storage CHAP ...........................................................76RHEL 5 RHEL 6 CHAP .............................................................77RHEL 4 CHAP ...................................................................................79

Internet Storage Name Server ..............................................................................80Microsoft iSNS ....................................................................80iSNS .....................................................................80iSNS iSCSI ..............................80

HP 3PAR StoreServ Storage .................................................................................80iSNS (RHEL ) .......................................................................81

7 FCoE ....................................................83Linux ...................................................................................................83FCoE .............................................................................................................83 BIOS FCoE ..................................................................................83

8 RHEL .....................................87HP 3PAR StoreServ Storage .................................................................87.......................................................................................................87

4

............................................................88LUN ..............................................................................................88..............................................................................................89Emulex HBA ...................................................................................89

LUN .........................................................................................89 1 - sysfs scan.........................................................................................................90 2 - ......................................................................................91

Emulex HBA .......................................91QLogic HBA ..................................................................................92

LUN .........................................................................................92 1 - echo sysfs ..................................................92 2 - add single device ...............................................................94

QLogic HBA .......................................95 iSCSI .................................................................96

RHEL 5 RHEL 6 ...............................................................96RHEL 4 .....................................................................................97

9 HP 3PAR ...........................................99...........................................................................................99............................................................................101......................................................................102Veritas Volume Manager ................................................................................106............................................................................107RHEL 6.x UNMAP .................................108

10 HP 3PAR StoreServ Storage ......................................111HP 3PAR StoreServ Storage .....................................................................111RHEL HBA BIOS .............................................................111

QLogic HBA HP 3PAR StoreServ Storage ..........................................111Emulex HBA HP 3PAR StoreServ Storage ..........................................111

RHEL Linux CD DVD .....................................................................112/etc/multipath.conf .......................................................................................113Emulex HBA ................................................................117 QLogic .............................................................................117

11 Veritas Cluster Server ..................................................................11912 RHEL Xen Virtualization ...............................................................12013 RHEL Cluster .................................................................12114 Red Hat Enterprise Virtualization (KVM/RHEV-H) ............................12215 Oracle Linux ..............................................................................123

RHEL Oracle Linux........................................................................123Unbreakable Oracle Linux ............................................................123Oracle Linux ................................................................................123

16 Oracle VM Server ................................................................12517 ......................................................................126

HP ..............................................................................................................126HP 3PAR .........................................................................................................126......................................................................................................................129HP 3PAR ...............................................................................................129

5

1Red Hat Enterprise Linux (RHEL) 4RHEL 5RHEL 6 Oracle Linux(OL) HP 3PAR StoreServ Storage HP 3PAR StoreServ Storage RHEL

HP 3PAR StoreServ Storage

: RHEL Oracle Linux

1 RHEL Oracle Linux

Oracle Linux RHEL

4.x4.x

5.x5.x

6.x6.x

HP 3PAR StoreServ Storage HP 3PAR StoreServ Storage HP

HP 3PAR StoreServ Storage Linux OS

(FC)

iSCSI

iSCSI

Fibre Channel over Ethernet (FCoE)HP 3PARStoreServ Storage RHEL HPSingle Point of Connectivity Knowledge (HP SPOCK) Web http://www.hp.com/storage/spock ()HP 3PAR Storage HP 3PAR Storage (6 )

2 HP 3PAR Storage

http://h20000.www2.hp.com/bizsupport/TechSupport/Home.jsp?lang=ja&cc=jp&prodTypeId=12169&prodSeriesId=5335712&lang=ja&cc=jp ()

HP 3PAR StoreServ 7000 Storage

http://h20000.www2.hp.com/bizsupport/TechSupport/Home.jsp?lang=ja&cc=jp&prodTypeId=12169&prodSeriesId=5157544&lang=ja&cc=jp ()

HP 3PAR StoreServ 10000 Storage

6

http://www.hp.com/storage/spockhttp://h20000.www2.hp.com/bizsupport/TechSupport/Home.jsp?lang=ja&cc=jp&prodTypeId=12169&prodSeriesId=5335712&lang=ja&cc=jphttp://h20000.www2.hp.com/bizsupport/TechSupport/Home.jsp?lang=ja&cc=jp&prodTypeId=12169&prodSeriesId=5335712&lang=ja&cc=jphttp://h20000.www2.hp.com/bizsupport/TechSupport/Home.jsp?lang=ja&cc=jp&prodTypeId=12169&prodSeriesId=5335712&lang=ja&cc=jphttp://h20000.www2.hp.com/bizsupport/TechSupport/Home.jsp?lang=ja&cc=jp&prodTypeId=12169&prodSeriesId=5157544&lang=ja&cc=jphttp://h20000.www2.hp.com/bizsupport/TechSupport/Home.jsp?lang=ja&cc=jp&prodTypeId=12169&prodSeriesId=5157544&lang=ja&cc=jphttp://h20000.www2.hp.com/bizsupport/TechSupport/Home.jsp?lang=ja&cc=jp&prodTypeId=12169&prodSeriesId=5157544&lang=ja&cc=jp

2 HP 3PAR Storage ()

http://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&

HP 3PAR Storage System

h_product=5044012&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jp ()


HP 3PAR StoreServ Software



HP 3PAR StoreServ Software


HP 3PAR OS HP 3PAR (HP 3PAR OS) HP Business Support Center (BSC) Web HP 3PAR Operating SystemUpgrade Pre-Planning Guidehttp://www.hp.com/go/bsc ()HP SPOCK Web http://www.hp.com/storage/spock ()


RHEL 4, RHEL 5, RHEL 6, or Oracle Linux HP 3PAR OS

HP 3PAR StoreServ Storage Red Hat Enterprise Linux or Oracle Linux RHEL (HBA) HP

: HBA HP 3PAR

HP 3PAR OS 7

http://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&h_product=5044012&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jphttp://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&h_product=5044012&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jphttp://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&h_product=5044012&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jphttp://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&h_product=5044012&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jphttp://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&h_product=5046476&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jphttp://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&h_product=5046476&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jphttp://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&h_product=5046476&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jphttp://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&h_product=5046476&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jphttp://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&h_product=5053605&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jphttp://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&h_product=5053605&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jphttp://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&h_product=5053605&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jphttp://h20180.www2.hp.com/apps/Nav?h_pagetype=s-001&h_lang=ja&h_cc=jp&h_product=5053605&h_client=S-A-R88-1&h_page=hpcom&lang=ja&cc=jphttp://www.hp.com/go/bschttp://www.hp.com/storage/spock

2 HP 3PAR StoreServ Storage

HP 3PAR StoreServ Storage RHEL HP 3PAR OS 3.1.xOS 2.3.x OS 2.2.x HP 3PAR StoreServ Storage HP 3PAR

HP 3PAR StoreServ Storage HP 3PAR StoreServ Storage (12 )

HP 3PAR OS 3.1.x 2.3.x HP 3PAR StoreServStorage

HP 3PAR OS 3.1.x 2.3.x HP 3PAR StoreServ Storage


: HP 3PAR Storage System FC HP Virtual Connect HP 3PAR StoreServ Storage BladeSystem c-Class HP Virtual ConnectFlexFabric 10Gb/24 HP 3PAR StoreServ Storage HP Virtual ConnectHP Virtual Connect HP Virtual ConnectHP Virtual Connect HP SAN DesignReference Guide HP BSC Web

http://www.hp.com/go/bsc ()

HP 3PAR StoreServ Storage HP 3PAR StoreServ Storage RHEL

1. HP 3PAR StoreServ Storage

a. controlport offline b. controlport config host -ct loop : -ct loop

c. controlport rst :

# controlport offline 1:5:1# controlport config host -ct loop 1:5:1# controlport rst 1:5:1


http://www.hp.com/go/bsc

2. HP 3PAR StoreServ Storage showport -par

loop point

# showport -par

N:S:P Connmode ConnType CfgRate MaxRate Class2 UniqNodeWwn VCN IntCoal0:0:1 disk loop auto 2Gbps disabled disabled disabled enabled0:0:2 disk loop auto 2Gbps disabled disabled disabled enabled0:0:3 disk loop auto 2Gbps disabled disabled disabled enabled0:0:4 disk loop auto 2Gbps disabled disabled disabled enabled0:4:1 host point auto 4Gbps disabled disabled disabled enabled0:4:2 host point auto 4Gbps disabled disabled disabled enabled0:5:1 host point auto 2Gbps disabled disabled disabled enabled0:5:2 host loop auto 2Gbps disabled disabled disabled enabled0:5:3 host point auto 2Gbps disabled disabled disabled enabled0:5:4 host loop auto 2Gbps disabled disabled disabled enabled1:0:1 disk loop auto 2Gbps disabled disabled disabled enabled1:0:2 disk loop auto 2Gbps disabled disabled disabled enabled1:0:3 disk loop auto 2Gbps disabled disabled disabled enabled1:0:4 disk loop auto 2Gbps disabled disabled disabled enabled1:2:1 host point auto 2Gbps disabled disabled disabled enabled1:2:2 host loop auto 2Gbps disabled disabled disabled enabled1:4:1 host point auto 2Gbps disabled disabled disabled enabled1:4:2 host point auto 2Gbps disabled disabled disabled enabled1:5:1 host loop auto 4Gbps disabled disabled disabled enabled1:5:2 host loop auto 4Gbps disabled disabled disabled enabled1:5:3 host loop auto 4Gbps disabled disabled disabled enabled1:5:4 host loop auto 4Gbps disabled disabled disabled enabled

HP 3PAR StoreServ Storage HP 3PAR StoreServ Storage

: HP 3PAR StoreServ Storage 2

HP 3PAR OS 3.1.x 2.3.x HP 3PAR StoreServ Storage 9

1. HP 3PAR StoreServ Storage showport -par

2. HP 3PAR OS CLI controlportoffline

# controlport offline 1:5:1

3. controlport config host -ct point -ct point

# controlport config host -ct point 1:5:1

4. controlport rst

# controlport rst 1:5:1

RHEL HP 3PAR StoreServ Storage HBA HP 3PAR StoreServ Storage Host Persona


1. createhost [options] [...]

# createhost -persona 1 redhathost 1122334455667788 1122334455667799

: HP 3PAR Host Explorer HP RHEL 4 Update 6 RHEL5.0 RHEL 6.0 Host Persona 1

Host Persona 1 Host_Explorer UARepLun 2 Host_Explorer Host Persona 1 SESLun

UARepLun VLUN LUN VLUN

RHEL UARepLun VLUN HP 3PAR OS 2.2.x Host Persona 6 2 Host Persona 6 1

: HP 3PAR Command Line Interface ReferenceHP 3PAR Management ConsoleHelpcontrolportcreatehost showhost

HP BSC Web http://www.hp.com/go/bsc ()

HP 3PAR OS 2.2.x HP 3PAR StoreServ Storage HP 3PAR OS 2.2.x HP 3PAR StoreServ Storage


HP 3PAR StoreServ Storage 1. HP 3PAR StoreServ Storage controlport persona 1 1 ::

2. controlport vcn disable -f 3. Persona

# showport -parN:S:P ConnTypeCfgRateClass2 VCN -----------Persona------------ IntCoal4:0:2 loop auto disable disable *(1) g_ven, g_hba, g_os, 0, DC enabled


HP 3PAR OS 2.2.x HP 3PAR StoreServ Storage 11


1 1. controlport persona

7 Port Persona 7 ::

2. controlport vcn disable -f 3. Persona

# showport -parN:S:P ConnType CfgRate Class2 VCN -----------Persona------------ IntCoal4:0:2 point auto disable disable *(7) g_ven, g_hba, g_os, 0, FA enabled

RHEL HP 3PAR StoreServ Storage HBA HP 3PAR StoreServ Storage

1. HP 3PAR StoreServ Storage

# createhost [options] []...

:

# createhost redhathost 1122334455667788 1122334455667799

2. showhost :

# showhost2 redhathost 1122334455667788 4:0:1

1122334455667799 5:0:1

HP 3PAR StoreServ Storage HP 3PAR StoreServ Storage HP 3PAR StoreServ Storage

: HP 3PAR Storage System FC HP Virtual Connect HP 3PAR StoreServ Storage BladeSystem c-Class HP Virtual Connect FLexFabric 10Gb/24 Virtual Connect SAN Fabric

HP Virtual ConnectHP Virtual Connect HP Virtual ConnectHP Virtual Connect HP SAN DesignReference Guide HP BSC Web





(Registered State Change Notifications,RSCN) World Wide Name (WWN) HP 3PARStoreServ Storage WWN WWN

HBA HP 3PAR StoreServStorage HBA HP 3PAR

1 1 1

1 1 (HBA ) HP 3PAR StoreServ Storage HBA HP

: /

: HP 3PAR StoreServ StorageHP 3PAR StoreServ Storage HP 3PAR HP

1 HP SANDesign Reference GuideBest PracticesZoning by HBAHP BSC Web http://www.hp.com/go/bsc ()HP

HBA HP 3PAR StoreServ Storage HP 3PAR OS CLI showhost

HP 3PAR HP 3PAR StoreServ Storage HP HP HP BSC Web HP SAN Design Reference Guidehttp://www.hp.com/go/bsc ()

13

http://www.hp.com/go/bschttp://www.hp.com/go/bsc


HBA HP 3PAR StoreServ Storage Brocade Brocade 3.0.2 Brocade 3xxx Brocade telnet portcfgshow

brocade2_1:admin> portcfgshowPorts 0 1 2 3 4 5 6 7-----------------+--+--+--+--+----+--+--+--Speed AN AN AN AN AN AN AN ANTrunk Port ON ON ON ON ON ON ON ONLocked L_Port .. .. .. .. .. .. .. ..Locked G_Port .. .. .. .. .. .. .. ..Disabled E_Port .. .. .. .. .. .. .. ..where AN:AutoNegotiate, ..:OFF, ??:INVALID.

FOS 6.3.1a Brocade 8G/s

admin> portcfgfillwordUsage: portCfgFillWord PortNumber Mode [Passive]Mode: 0/-idle-idle - IDLE in Link Init, IDLE as fill word (default)

1/-arbff-arbff - ARBFF in Link Init, ARBFF as fill word2/-idle-arbff - IDLE in Link Init, ARBFF as fill word (SW)3/-aa-then-ia - If ARBFF/ARBFF failed, then do IDLE/ARBFF

HP 3 (aa-then-ia) 3 portcfgfillword 8G HBA portstatsshow er_bad_os (invalid ordered set) 8G HBAARBFF-ARBFF 3 4Gb/2GbHBA HBA Fabric OScommand Reference Manual supporting FOS 6.3.1a FOS

HP (HP SN8000B 8-slot SAN HPSN8000B 4-slot SAN HP SN6000B 16Gb FC HP SN3000B 16Gb FC ) 3

McDATA ( [G] [GX-port])

Cisco HP 3PAR StoreServ Storage HBA[AdminMode = FX][AdminSpeed = auto port] [auto negotiate]

QLogic [GL-port] auto-detect HP 3PAR StoreServ Storage QLogic I/OStream Guard [disable] [auto] [enable]


I/O

HP 3PAR StoreServ Storage 64HP 3PARStoreServ Storage 1,024

HP 3PAR StoreServ Storage HBA I/O

QLogic 2G: 497 LSI 2G: 510 Emulex 4G: 959 HP 3PAR HBA 4G: 1638 HP 3PAR HBA 8G: 3276 (HP 3PAR StoreServ 10000 HP 3PAR StoreServ 7000)

I/O HBA

HBA I/O HP 3PAR StoreServ Storage target queue full I/O I/O HP 3PAR StoreServ Storage

: HP 3PAR StoreServ Storage

HP 3PAR Priority OptimizationHP 3PAR OS 3.1.2.MU2 HP 3PAR Priority Optimization I/O Priority Optimization Priority Optimization () HP 3PAR http://www.hp.com/go/bsc () HP 3PAR Priority Optimization

: HP 3PAR OS 3.1.2

() HP 3PAR Storage I/O Fibre Channel HP 3PAR Storage ID ID ( WWN) NPIV

15


ID I/O () HP 3PAR OS 3.1.2

HP 3PAR Storage 2 HP 3PAR Storage 0:1:10:5:1 1:1:11:5:1 0 0:1:10:5:1 1:1:11:5:1 1:1:11:5:1

2 HP 3PAR Storage 0 0 1 2 2 3 1 1 0 3 3 2 I/O

HP 3PAR Command Line Interface Administrators Manual

HP 3PAR NPIV HP 3PAR ID ID ID

showport :: Partner FailoverState FailoverState N:S:P Partner 2 FailoverState

none:

failover_pending:

failed_over:

active:

active_down:

failback_pending:


HP 3PAR CLI showport showport Partner 1:1:1 0:1:1 0:1:1 1:1:1 FailoverState none

showport FailoverState 1 1:1:1 0:1:1 1:1:1 I/O 0:1:1

showhost

# showhostId Name Persona ---------------WWN/iSCSI_Name--------------- Port1 server1 Generic 5001438009AE770E 0:1:1

5001438009AE770C 0:1:15001438009AE770E 1:1:15001438009AE770C 1:1:1

showhost Port HBA 0:1:1 1 1 2

# showhostId Name Persona ---------------WWN/iSCSI_Name--------------- Port1 server1 Generic 5001438009AE770E 0:1:1

5001438009AE770C 0:1:15001438009AE770E 0:1:15001438009AE770C 0:1:1

FailoverState none

17

0 GUI I/O

controlport failover controlport failback

http://h20195.www2.hp.com/V2/GetPDF.aspx/4AA4-4545ENW.pdf ()

iSCSI FCoE

FCoE-to-FC FCoE-to-FC

1 FCoE FC


http://h20195.www2.hp.com/V2/GetPDF.aspx/4AA4-4545ENW.pdf

2 FCoE FCoE

RHEL (FCoE ) FCoE FC HP 3PAR StoreServ Storage (FC )

FCoE-to-FC 19

3 HP 3PAR StoreServ Storage iSCSI iSCSI

iSCSI HP 3PAR StoreServ Storage iSCSI

: HP 3PAR StoreServ Storage iSCSI iSCSI

1. HP 3PAR StoreServ 10000 HP 3PAR StoreServ 7000 10 Gb iSCSI controlport (HP 3PARV T F ) showport showport -i :

# showportN:S:P Mode State ----Node_WWN---- -Port_WWN/HW_Addr- Type Protocol0:3:1 suspended config_wait - - cna -0:3:2 suspended config_wait - - cna -

# showport -iN:S:P Brand Model Rev Firmware Serial HWType0:3:1 QLOGIC QLE8242 58 0.0.0.0 PCGLT0ARC1K3SK CNA0:3:2 QLOGIC QLE8242 58 0.0.0.0 PCGLT0ARC1K3SK CNA

2. State=config_wait Firmware=0.0.0.0 controlport configiscsi showport showport -i

:

# controlport config iscsi 0:3:1# controlport config iscsi 0:3:2# showportN:S:P Mode State ----Node_WWN---- -Port_WWN/HW_Addr- Type Protocol...0:3:1 target ready - 2C27D7521F3E iscsi iSCSI0:3:2 target ready - 2C27D7521F3A iscsi iSCSI# showport -i...N:S:P Brand Model Rev Firmware Serial HWType...0:3:1 QLOGIC QLE8242 58 4.8.76.48015 PCGLT0ARC1K3U6 CNA0:3:2 QLOGIC QLE8242 58 4.8.76.48015 PCGLT0ARC1K3U6 CNA

3. showport -iscsi iSCSI

20 HP 3PAR StoreServ Storage iSCSI

4. controliscsiport addr [-f] iSCSI IP

# controliscsiport addr 10.100.0.101 255.255.0.0 -f 0:3:1# controliscsiport addr 10.100.0.201 255.255.0.0 -f 1:3:1

5. showport -iscsi

: VLAN iSCSI iSCSI (53 )

6. controliscsiport ping HP 3PARStoreServ Storage iSCSI iSCSI

# controliscsiport ping 10.100.0.100 0:3:1Ping succeeded

: HP 3PAR OS IP controliscsiport gw [-f] HP 3PAR OS

iSCSI iSCSI iSCSI RHEL 5 RHEL 6 iSCSI (58 )

iSCSI 21

: /etc/sysctl.conf

net.ipv4.conf.all.arp_filter = 1

: iSCSI iSCSI /iSCSI HP 3PARStoreServ Storage iSCSI RHEL 5 RHEL 6 iscsiadm iSCSI (72 ) iSCSI

# cat /etc/iscsi/initiatorname.iscsi

Initiator Name= iqn.1994-05.com.redhat:a3df53b0a32dS

iSCSI BIOS [ Ctrl-S ] hbacmd ocmanager UI

# hbacmd GetInitiatorProperties 28-92-4a-af-f5-61

Initiator login options for 28-92-4a-af-f5-61:

Initiator iSCSI Name: iqn.1990-07.com.emulex:28-92-4a-af-f5-61

Web OneCommandManager Command Line Interface Version 6.1 UserManualiSCSI Commandshttp://www-dl.emulex.com/support/elx/r32/b16/docs/apps/ocm_cli_manual_elx.pdf ()

1. 10 G iSCSI Emulex OneCommand Manager /usr/sbin/ocmanager/hbacmd QLogic QConvergeConsole Manager /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli 10 GbCNA MAC 10 Gb NIC IP

: iSCSI HPNC551/553/FlexFabric 554/CN1100E iSCSI HP NC551

HP NC553

HP FlexFabric 554

HP CN1100E

:

qaucli 10 Gb CNA MAC 10 Gb NIC IP


http://www-dl.emulex.com/support/elx/r32/b16/docs/apps/ocm_cli_manual_elx.pdf

2. iSCSI iSCSI HP 3PAR OS CLI showhost

# showhostId Name Persona ----------WWN/iSCSI_Name----------- Port-- iqn.1994-05.com.redhat:a3df53b0a32d 0:3:1

iqn.1994-05.com.redhat:a3df53b0a32d 1:3:1

: HP 3PAR Host Explorer HP RHEL 4 Update 6 RHEL 5.0 RHEL 6.0 Host Persona 1

HP 3PAR OS 2.2.x Host Persona 6 Host Persona 6 Host Persona 1

HP 3PAR Host Explorer UARepLun 2 HP 3PAR Host ExplorerHost Persona 1 SESLun UARepLun VLUN LUN VLUN RHEL UARepLun VLUN

3. HP 3PAR OS CLI createhost -iscsi iSCSI HP 3PAR OS 3.1.x OS 2.3.x HP 3PAR StoreServ Storage createhost -persona1 :

# createhost -iscsi -persona 1 redhathost iqn.1994-05.com.redhat:a3df53b0a32d

: RHEL 4 iSCSI

# cat /etc/iscsi/initiatorname.iscsiInitiatorName=iqn.1987-05.com.cisco:01.4766d09183f3

4. HP 3PAR OS 2.3.x OS 3.1.x HP 3PAR

# showhostId Name Persona ----------WWN/iSCSI_Name----------- Port0 redhathost Generic iqn.1994-05.com.redhat:a3df53b0a32d 0:3:1


: HP 3PAR OS 2.2.x HP 3PAR StoreServ Storage Personashowhost

HP 3PAR OS 2.2.x HP 3PAR StoreServ Storage showhost

# showhostId Name -----------WWN/iSCSI_Name------------ Port0 linux iqn.1994-05.com.redhat:a3df53b0a32d 0:3:1

iSCSI 23


RHEL iscsiadm iscsiadm iSCSI

SendTargets iSCSI Discovery

# iscsiadm -m discovery -t sendtargets -p 10.0.0.10:3260

iSCSI

iSCSI

iSCSI

# iscsiadm -m node --logoutall=all

iSCSI

iSCSI

# iscsiadm -m node -o new -p 10.0.0.30:3260

iSCSI

SendTarget iSCSI

# iscsiadm -m discovery -o delete -p 10.0.0.10


iSCSI

# iscsiadm -m node -T iqn.2000-05.com.3pardata:21110002ac0001a6 -p 10.0.0.20:3260

# iscsiadm -m discovery

# iscsiadm -m discovery -p 10.0.0.10:3260

# iscsiadm -m session -r 1 --stats

# iscsiadm -m session

iSCSI LUN

# iscsiadm -m session -R

I/O


QLogic 1G: 512 QLogic 10G: 2048 (HP 3PAR StoreServ 10000 HP 3PAR StoreServ 7000 )

I/O HBA


HP 3PAR Priority OptimizationHP 3PAR OS 3.1.2.MU2 HP 3PAR Priority Optimization

25

I/O Priority Optimization Priority Optimization () HP 3PAR http://www.hp.com/go/bsc () HP 3PAR Priority Optimization



4 FCoE HP 3PAR StoreServ Storage FCoE FCoE FCoE

Linux FCoE HP 3PAR StoreServ Storage FCoE FCoE

: FCoE VLAN VLAN

1. HP 3PAR StoreServ 10000 HP 3PAR StoreServ 7000 CNA controlport (HP 3PAR T F ) FCoE :

# showportN:S:P Mode State ----Node_WWN---- -Port_WWN/HW_Addr- Type Protocol0:3:1 suspended config_wait - - cna -0:3:2 suspended config_wait - - cna -

# showport -iN:S:P Brand Model Rev Firmware Serial HWType0:3:1 QLOGIC QLE8242 58 0.0.0.0 PCGLT0ARC1K3U4 CNA0:3:2 QLOGIC QLE8242 58 0.0.0.0 PCGLT0ARC1K3U4 CNA

2. State=config_wait Firmware=0.0.0.0 controlport configfcoe showport showport -i

:

# controlport config fcoe 0:3:1# controlport config fcoe 0:3:2# showport 0:3:1 0:3:2N:S:P Mode State ----Node_WWN---- -Port_WWN/HW_Addr- Type Protocol LabelPartner FailoverState0:3:1 target ready 2FF70002AC000121 20310002AC000121 host FCoE -

- -0:3:2 target ready 2FF70002AC000121 20320002AC000121 free FCoE -

- -# showport -i 0:3:1 0:3:2N:S:P Brand Model Rev Firmware Serial HWType0:3:1 QLOGIC QLE8242 58 4.11.122 PCGLT0ARC1K3U4 CNA0:3:2 QLOGIC QLE8242 58 4.11.122 PCGLT0ARC1K3U4 CNA

3. showport -fcoe FCoE :

# showport -fcoeN:S:P ENode_MAC_Address PFC_Mask0:3:1 00-02-AC-07-01-21 0x080:3:2 00-02-AC-06-01-21 0x00

FCoE FCoE FCoE 27

: iSCSI FCoE 1. showport

# showport

0:3:1 target ready - 000E1E05BEE6 iscsi iSCSI - - -

0:3:2 target ready - 000E1E05BEE2 iscsi iSCSI - - -

2. iSCSI



showport

0:3:1 target offline - 000E1E05BEE2 iscsi iSCSI0:3:2 target offline -

000E1E05BEE2 iscsi iSCSI

3. FCoE

# controlport config fcoe 0:3:1

# controlport config fcoe 0:3:2

controlport rst 0:3:1

controlport rst 0:3:2

0:3:1 target offline - 000E1E05BEE2 iscsi iSCSI0:3:2 target offline -

000E1E05BEE2 iscsi iSCSI

showport

0:3:1 target ready 2FF70002AC000121 20310002AC000121 host FCoE

- - -

0:3:2 target ready 2FF70002AC000121 20320002AC000121 free FCoE

- - -

4. showport -fcoe FCoE :

# showport -fcoe

N:S:P ENode_MAC_Address PFC_Mask

0:3:1 00-02-AC-07-01-21 0x08

0:3:2 00-02-AC-06-01-21 0x00

I/O


QLogic CNA: 1748 (HP 3PAR StoreServ 10000 HP 3PAR StoreServ 7000 ) I/O HBA

28 FCoE HP 3PAR StoreServ Storage


: HP 3PAR StoreServ Storage

HP 3PAR Priority OptimizationHP 3PAR OS 3.1.2.MU2 HP 3PAR Priority Optimization I/O Priority Optimization Priority Optimization () HP 3PAR http://www.hp.com/go/bsc () HP 3PAR Priority Optimization

HP 3PAR Priority Optimization 29


5

: RHEL 6.x RHEL 5.x RHELOS RHEL 4RHEL 5 RHEL 6

Emulex gcc Developmental Tool RHEL RHEL CD gcc gcc RHEL 4 Update 6 Linux gcc

# rpm -qa | grep gccgcc-java-3.4.6-9gcc-3.4.6-9compat-gcc-32-c++-3.2.3-47.3gcc-c++-3.4.6-9compat-libgcc-296-2.96-132.7.2libgcc-3.4.6-9gcc-g77-3.4.6-9libgcc-3.4.6-9

Emulex HBA Emulex (CNA) HBA CNA

Emulex

: HP Web Emulex

http://www8.hp.com/jp/ja/support.html ()RHEL Emulex /etc/modprobe.conf ramdisk (32 ) () Emulex Web Emulex

RHEL Emulex Emulex 1. Emulex Web

www.emulex.com ()2. tar xvzf lpfc__driver_kit-.tar.gz

:

# tar xvzf lpfc_2.6_driver_kit-8.2.0.29-1.tar.gzlpfc_2.6_driver_kit-8.2.0.29-1/

30

http://www8.hp.com/jp/ja/support.htmlwww.emulex.com

lpfc_2.6_driver_kit-8.2.0.29-1/lpfcdriver_2.6-8.2.0.29-1.noarch.rpmlpfc_2.6_driver_kit-8.2.0.29-1/lpfc-installlpfc_2.6_driver_kit-8.2.0.29-1/README

3. cd lpfc_ _driver_kit- :

# cd lpfc_2.6_driver_kit-8.2.0.29-1

4. lpfc-install lpfc README

# ./lpfc-install

a. rpm lpfcdriver-_ /usr/src/lpfc :

# ls /usr/src/lpfc/lpfcdriver*lpfcdriver-2.6-8.0.16.40-2

b. lpfc /etc/modprobe.conf c. Emulex lpfc.ko /lib/modules//kernel/drivers/scsi/lpfc lpfc /usr/src/lpfc/savedfiles

d. ramdisk ramdisk /boot/initrd-.img

: ramdisk initrd-.img ramdisk : =2.6.18-53.el5 ramdisk initrd.2.6.18-53.el5.img

: /etc/modprobe.conf.local Emulex HP 3PARStoreServ Storage

lpfc-install HBA /etc/modprobe.conf

# cat /etc/modprobe.confalias eth0 e1000alias eth1 e1000alias scsi_hostadapter mptbasealias scsi_hostadapter1 mptscsihalias usb-controller ehci-hcdalias usb-controller1 uhci-hcd

Emulex HBA 31

alias scsi_hostadapter2 lpfcalias scsi_hostadapter3 lpfc

/etc/modprobe.conf ramdisk /etc/modprobe.conf Emulex HBA ramdisk 1. ramdisk RHEL HBA /etc/modprobe.conf HBA / RHEL 6

: /etc/modprobe.conf RHEL 6 ramdisk

1. /etc/modprobe.d/modprobe.conf 2. HP 3PAR HP 3PAR OS 3.1.1

options lpfc lpfc_devloss_tmo=14 lpfc_lun_queue_depth=16

lpfc_discovery_threads=32

RHEL 5

options lpfc lpfc_devloss_tmo=14 lpfc_lun_queue_depth=16lpfc_discovery_threads=32

RHEL 4

options lpfc lpfc_nodev_tmo=14 lpfc_lun_queue_depth=16lpfc_discovery_threads=32

: HP 3PAR HP 3PAR OS 3.1.1 RHEL lpfc_devloss_tmo lpfc_nodev_tmo 14 1

2. OS LUN SCSI /etc/modprobe.conf

: RHEL 6.x /etc/modprobe.conf RHEL6

256 LUN OS

options scsi_mod max_luns=256

32

: modprobe.conf SCSI ramdisk SCSI SCSI (sdasdb) SCSI sda scsi_hostadapter /etc/modprobe.conf

3. /etc/modprobe.conf HP 3PAR OS 3.1.1 HP 3PAR RHEL 6.x RHEL 5.x HP 3PAR HP 3PAR OS (OS 3.1.1) lpfc_devloss_tmo 1

# cat /etc/modprobe.confalias eth0 e1000alias eth1 e1000alias scsi_hostadapter mptbasealias scsi_hostadapter1 mptscsihalias usb-controller ehci-hcdalias usb-controller1 uhci-hcdalias scsi_hostadapter2 lpfcalias scsi_hostadapter3 lpfcoptions lpfc lpfc_devloss_tmo=14 lpfc_lun_queue_depth=16lpfc_discovery_threads=32options scsi_mod max_luns=256

HBA HP 3PAR StoreServ Storage I/O I/O QUEUE FULL SCSI LUN Emulex lpfc_hba_queue_depth LUN lpfc_lun_queue_depth

HP 3PAR StoreServ Storage LUN SCSI

http://h20195.www2.hp.com/v2/GetDocument.aspx?docname=4AA4-5094ENW&doctype=white%20paper&doclang=EN_US&searchquery=Storage|3par%20&cc=us&lc=en()

: /etc/modprobe.conf ramdisk ramdisk

Emulex HBA 33

http://h20195.www2.hp.com/v2/GetDocument.aspx?docname=4AA4-5094ENW&doctype=white%20paper&doclang=EN_US&searchquery=Storage|3par%20&cc=us&lc=enhttp://h20195.www2.hp.com/v2/GetDocument.aspx?docname=4AA4-5094ENW&doctype=white%20paper&doclang=EN_US&searchquery=Storage|3par%20&cc=us&lc=en

4. ramdisk RHEL 4 RHEL 5 mkinitrd ramdisk

# /sbin/mkinitrd -v -f /boot/

Oracle UEK 5.7 mkinitrd

# /sbin/mkinitrd --builtin=ehci-hcd --builtin=ohci-hcd --builtin=uhci-hcd -f-v /boot/initrd-2.6.32-200.13.1.el5uek.img 2.6.32-200.13.1.el5uek

RHEL 6 dracut ramdisk

# /sbin/dracut -v -f /boot/

ramdisk

# /sbin/dracut -v -f /boot/initrd-2.6.18-53.el5.img 2.6.18-53.el5Creating initramfsLooking for deps of module scsi_modLooking for deps of module sd_mod scsi_modLooking for deps of module scsi_transport_spi: scsi_mod. . .copy from `/lib/modules/2.6.18-8.el5/kernel/drivers/scsi/scsi_transport_fc.ko'[elf64-x86-64] to `/tmp/initrd.l13681/lib/scsi_transport_fc.ko' [elf64-x86-64]copy from `/lib/modules/2.6.18-8.el5/kernel/drivers/scsi/lpfc/lpfc.ko' [elf64-x86-64] to `/tmp/initrd.l13681/lib/lpfc.ko' [elf64-x86-64]. . .Loading module jbdLoading module ext3Loading module scsi_modLoading module scsi_mod with options max_luns=256Loading module sd_modLoading module mptbaseLoading module mptscsihLoading module scsi_transport_fcLoading module lpfc with options lpfc_topology=0x02 lpfc_devloss_tmo=14lpfc_lun_queue_depth=16 lpfc_discovery_threads=32

5. grub /etc/grub.conf /boot/grub/grub.confinitrd ramdisk

# vi /etc/grub.confdefault=timeout=5hiddenmenuftitle RedHat Enterprise Linux Server (2.6.18-8.el5)root (hd0,2)kernel /boot/vmlinuz-2.6.18-8.el5 ro root=LABEL=/ rhgb quietinitrd /boot/initrd-2.6.18-8.el5.img

34

Emulex HBA NVRAM BIOS Emulex HBA NVRAM BIOS Emulex Lightpulse BIOS NVRAM BIOS

: Emulex HBA NVRAM HBA HBA HBA

SAN 1. BIOS [Alt] [Ctrl]5 [E]

: HBA HBA

2. 3. [Main] [Enable/Disable Boot from SAN] 1 SAN

:

1. [Configure Boot Devices] 8

2. 3. []

4. LUN LUN 0255

: 256 LUN 16 LUN B#D B#W DID WWPN B1D 1 DID B2W 2 WWPN

5. 2 6. WWPN WWPN DID

7. [Esc] BIOS 8. Emulex Boot Code User Manual

Emulex HBA 35

HBACMD Emulex HBA HBACMD Emulex HBA

Emulex HBA CLI (OneCommand) GUI HBA

WWPN HBA WWPN

# hbacmd ListHBAs

Manageable HBA List

Port WWN : 10:00:00:00:c9:69:d6:ccNode WWN : 20:00:00:00:c9:69:d6:ccFabric Name : 00:00:00:00:00:00:00:00Flags : 8000fe0dHost Name : dl360g7-16.3pardata.comMfg : Emulex CorporationSerial No. : VM72838048Port Number : 0Mode : InitiatorPCI Function : 0Port Type : FCModel : LPe11002-M4

Port WWN : 10:00:00:00:c9:69:d6:cdNode WWN : 20:00:00:00:c9:69:d6:cdFabric Name : 00:00:00:00:00:00:00:00Flags : 8000fe0dHost Name : dl360g7-16.3pardata.comMfg : Emulex CorporationSerial No. : VM72838048Port Number : 1Mode : InitiatorPCI Function : 1Port Type : FCModel : LPe11002-M4

BIOS

# hbacmd EnableBootCode

hbacmd Web Emulex OneCommand Manager

QLogic HBA HBA QLogic HBA

36

QLogic

: RHEL QLogic /etc/modprobe.conf ramdisk (37 )

QLogic 1. (RHEL SANsurfer Linux )

www.qlogic.com 2. README

/etc/modprobe.conf ramdisk

: /etc/modprobe.conf RHEL 6 ramdisk /etc/modprobe.d/modprobe.conf

1. HP 3PAR HP 3PAR OS 3.1.1 options qla2xxx qlport_down_retry=10 3PAR HP 3PAR OS 3.1.1 /etc/modprobe.conf

: HP 3PAR OS 3.1.1 HP 3PAR OS qlport_down_retry 10 1

# cat /etc/modprobe.confalias scsi_hostadapter1 qla2xxxoptions scsi_mod max_luns=256options qla2xxx ql2xmaxqdepth=16 qlport_down_retry=10 ql2xloginretrycount=30

HP 3PAR StoreServ Storage I/O I/O QUEUEFULL SCSI LUN QLogic () FFFF (65535) ( BIOS 32 )LUN 32 () ql2xmaxqdepth LUN QLogic

HP 3PAR OS 3.1.1 HP 3PAR RHEL ql2xmaxqdepth 16 /etc/modprobe.conf

# cat /etc/modprobe.confalias scsi_hostadapter1 qla2xxxalias scsi_hostadapter2 qla2300alias scsi_hostadapter3 qla2322alias scsi_hostadapter4 qla2400alias scsi_hostadapter5 qla6312options scsi_mod max_luns=256options qla2xxx qlport_down_retry=10 ql2xloginretrycount=30 ql2xmaxqdepth=16ConfigRequired=0install qla2xxx /sbin/modprobe --ignore-install qla2xxxremove qla2xxx /sbin/modprobe -r --first-time --ignore-remove qla2xxx

QLogic HBA 37

HP 3PAR StoreServ Storage LUN SCSI

2. /etc/modprobe.conf ramdisk

3. mkinitrd QLogic

# mkinitrd -f -v /boot/initrd-.img

:

# mkinitrd -f -v /boot/initrd-2.6.18-53.el5.img 2.6.18-53.el5

: RHEL 6 dracut ramdisk

# /sbin/dracut -v -f /boot/

4. 2 ramdisk

:

Creating initramfs. . . .. . . .Looking for deps of module scsi_modLooking for deps of module sd_mod: scsi_mod. . . .. . . .Looking for deps of module qla2xxx: intermodule scsi_modLooking for deps of module intermodule. . . .. . . .

grub /etc/grub.conf /boot/grub/grub.conf initrd ramdisk

# vi /etc/grub.confdefault=timeout=5hiddenmenutitle RedHat Enterprise Linux Server (kernel name)root (hd0,0)kernel / ro root=LABEL=/ rhgb quietinitrd /

38

QLogic HBA NVRAM BIOS QLogic HBA NVRAM BIOS QLogic Fast!UTIL QLogic 23xx24xx 25xx NVRAM Fast!UTIL

: QLogic HBA NVRAM HBA HBA HBA NVRAM

1. Fast!UTIL [ Ctrl-Q ]

: HBA

2. 3. HBA [Configuration Settings]+[Restore

Default Settings]4.

: QLogic HBA

[Configuration Settings][Advanced Adapter Settings][Execution Throttle: 256]

[Configuration Settings][Advanced Adapter Settings][LUNs per Target: 256]

[Configuration Settings][Extended Firmware Settings][Data Rate: 2 (AutoNegotiate)]5.

[ConfigurationSettings][Connection Options: 0 (Loop Only)]

[ConfigurationSettings][Connection Options: 1 (Point to Point Only)]

6. HBA

SCLI QLogic HBA SCLI QLogic HBA

: QLogic Fast!Util HBA QLogic

: Itanium Intel SCLI Fast!Util HBA BIOS

Extensible Firmware Interface (EFI) (BIOS) IntelItanium QLogic QLogic SANsurfer FC CLI SCLI QLogic Web

QLogic HBA 39

HBA QLogic SANsurfer FC CLI

# /opt/QLogic_Corporation/SANsurferCLI/scli -n X CO 1

# /opt/QLogic_Corporation/SANsurferCLI/scli -n X CO 0

X HBA FC HBA 0 HBA 1 3 /

# /opt/Qlogic_Corporation/SANsurferCLI/scli -n 1 CO 1# /opt/Qlogic_Corporation/SANsurferCLI/scli -n 3 CO 1

HBA 1 3

# /opt/Qlogic_Corporation/SANsurferCLI/scli -n 1 CO 0# /opt/Qlogic_Corporation/SANsurferCLI/scli -n 3 CO 0

# /opt/Qlogic_Corporation/SANsurferCLI/scli -I 1# /opt/Qlogic_Corporation/SANsurferCLI/scli -I 3

SANsurfer FCCLI

LUNs per Target: 256

Data Rate: 4 (Auto Negotiate)

Brocade HBA HBA Brocade (HBA)

Brocade

: Brocade RHEL Brocade Brocade HBA NVRAM BIOS (41 )

RHEL Brocade Brocade

40

1. www.brocade.com () tar xvzfbrocade_driver_linux-.tar.gz :

# tar zxvf brocade_driver_linux_rhel6_v3-2-1-0.tar.gzbfa_driver_linux-3.2.1.0-0.noarch.rpmbfa_util_linux_noioctl-3.2.1.0-0.noarch.rpmbna_driver_linux-3.2.1.0-0.noarch.rpmbna-snmp-3.2.1.0-rhel6.i386.rpmbna-snmp-3.2.1.0-rhel6.x86_64.rpmbrocade_install_rhel.shbrocade_install.shdriver-bld-info.xmlRHEL60/RHEL61/RHEL62/RHEL62/kmod-bna-3.2.1.0-0.el6.x86_64.rpmRHEL62/kmod-bfa-3.2.1.0-0.el6.i686.rpmRHEL62/kmod-bfa-3.2.1.0-0.el6.ppc64.rpmRHEL62/kmod-bna-3.2.1.0-0.el6.i686.rpmRHEL62/kmod-bna-3.2.1.0-0.el6.ppc64.rpmRHEL62/kmod-bfa-3.2.1.0-0.el6.x86_64.rpmRHEL63/RHEL63/kmod-bna-3.2.1.0-0.el6.x86_64.rpmRHEL63/kmod-bfa-3.2.1.0-0.el6.i686.rpmRHEL63/kmod-bna-3.2.1.0-0.el6.i686.rpmRHEL63/kmod-bfa-3.2.1.0-0.el6.x86_64.rpmRHEL64/RHEL64/kmod-bna-3.2.1.0-0.el6.x86_64.rpmRHEL64/kmod-bfa-3.2.1.0-0.el6.i686.rpmRHEL64/kmod-bna-3.2.1.0-0.el6.i686.rpmRHEL64/kmod-bfa-3.2.1.0-0.el6.x86_64.rpm.sh

2. brocade_install.sh bfa

# ./brocade_install.shInstalling the Brocade driver 3.2.1.0 RPM'sinitrd backup completeBackup file name : initramfs-2.6.32-279.el6.x86_64.img.bakInstalling the BFA driver RPM: bfa_driver_linux-3.2.1.0-0.noarch.rpmPreparing... ########################################### [100%]1:bfa_driver_linux ########################################### [100%]

Building bfa driver ................ doneinitrd update .... doneInstalling the util driver RPMPreparing... ###########################################

[100%]1:bfa_util_linux_noioctl ########################################### [100%]

Install cli ... doneInstall HBAAPI library ... doneInstall HBAAGENT ... doneLoading bfa driver ... doneinitrd update .... done

Brocade HBA NVRAM BIOS Brocade HBA NVRAM BIOS

Brocade HBA 41

www.brocade.com

Brocade BIOS NVRAM BIOS

: Brocade HBA NVRAM HBA HBA HBA

SAN BIOS [Alt] [Ctrl] 5 [B]

: HBA HBA

1. [Adapter Settings] BIOS [Enabled] 2. [Alt-S] 3. [Return to Brocade Config

Menu] [Exit Brocade Config Menu]

:

1. [Boot Device Settings] 2. [Boot Device] [ENTER] 3. [Boot Target] [ENTER] 4. [Boot LUN] [ENTER] 5. [Boot Device] 234 6. [Alt-S] 7. [Return to Brocade

Config Menu] [Exit Brocade Config Menu]

BCU Brocade HBA BCU Brocade HBA Brocade HBA CLI GUI HBA

BCU CLI GUI www.brocade.com () Brocade FC HBA Path TOV 30 14 Brocade BCU :1. bcu port --list

# bcu port --list-------------------------------------------------------------Port# FN Type PWWN/MAC FC Addr/ Media State Spd

Eth dev-------------------------------------------------------------1/0 - fc 10:00:8c:7c:ff:30:41:60 036100 sw Linkup 4G

0 fc 10:00:8c:7c:ff:30:41:60 036100 sw Linkup 4G1/1 - fc 10:00:8c:7c:ff:30:41:61 036000 sw Linkup 4G

42

http://www.brocade.com/

1 fc 10:00:8c:7c:ff:30:41:61 036000 sw Linkup 4G-------------------------------------------------------------

2. bcu fcpim --pathtov path_tov

# bcu fcpim --pathtov 1/0 14path timeout is set to 14

SCSI HP 3PAR StoreServ Storage RHEL SCSI RHEL

RHEL 6: SCSI 30

RHEL 5: SCSI 60

RHEL 4: SCSI 30 60

! RHEL 4 RHEL 5 : 60 HP 3PAR StoreServStorage SCSI Remote Copy SCSI 60 60

UDEV SCSI RHEL 4 udev SCSI 30 60 HP 3PAR udev SCSI (/dev/sda) SCSI

: udev RHEL Update 5 RHEL 6 () RHEL 4 Update 4 udev QLogic SCSI (45 ) ql_ch_scsi_timeout SCSI

1. udev RHEL CD :

# rpm -qa | grep udevudev-039-10.19.el4.x86_64.rpm

SCSI 43

2. udev rules 56_3par_timeout.rules /etc/udev/rules.d

/etc/udev/rules.d/56-3par.timeout.rules

KERNEL="sd*[!0-9]", SYSFS{vendor}="3PARdata", PROGRAM="/bin/sh -c 'echo 60 >/sys/block/%k/device/timeout'" NAME="%k"

56-3par-timeout.rules 2 udev 56-3par-timeout.rules 51-by-id.rules udev 56-3par-timeout.rules UDEV SCSI (43 ) 56-3par-timeout.rulesudev

# ls /etc/udev/rules.d/. . . . .40-multipath.rules50-udev.rules51-by-id.rules56-3par-timeout.rules

SCSI HP PAR udev RHEL (87 )

# udevinfo -a -p /sys/block/sdx

:

# udevinfo -a -p /sys/block/sdn |grep timeoutSYSFS{timeout}="60"

RHEL 6 SCSI

cat /sys/class/scsi_device/*/device/timeout

Emulex HBA RHEL 5

/sys/class/scsi_device/*/device/timeout

HP 3PAR StoreServ Storage udev udevstart udev 60 udevstart I/O

# udevstart

44

udev

QLogic SCSI QLogic HBA LUN SCSI 60 udev

LUN SCSI 30

Script ql_ch_scsi_timeout.sh

qlogicname="/sys/class/scsi_host"timeout=60ls $qlogicname | grep "[0-9][0-9]*" | while read linedofname=${qlogicname}/$linecurr=`pwd`cd $fnamefind . -follow -name "timeout" | grep -v "generic" | while read line2dovendorcheck=`cat ${line2%timeout}vendor | grep -c "3PARdata"`if [ $vendorcheck -gt 0 ] ; thenecho "modifying file: [$fname$line2]"echo "$timeout" > $line2fidonecd $currdone

/etc/rc.local OS /etc/rc.local 777

/etc/rc.local

# cat /etc./rc.local!/bin/sh## This script will be executed *after* all the other init scripts.# You can put your own initialization stuff in here if you don't# want to do the full Sys V style init stuff.touch /var/lock/subsys/localqlogicname="/sys/class/scsi_host"timeout=60ls $qlogicname | grep "[0-9][0-9]*" | while read linedofname=${qlogicname}/$linecurr=`pwd`cd $fnamefind . -follow -name "timeout" | grep -v "generic" | while read line2dovendorcheck=`cat ${line2%timeout}vendor | grep -c "3PARdata"`if [ $vendorcheck -gt 0 ] ; thenecho "modifying file: [$fname$line2]"echo "$timeout" > $line2fidone

SCSI 45

cd $currdone

Emulex SCSI udev set_target_timeout.sh SCSI 30 60 http://www.emulex.com() Emulex Linux Web [Linux Tools] I/O

: Emulex

# ./set_target_timeout.sh

# ls /sys/class/scsi_host/host2/device

target2:0:0

SCSI 2 0

# ./set_target_timeout 2 0 60modifying device /sys/class/scsi_device/2:0:0:0/devicefound timeout at value 30new timeout value is 60modifying device /sys/class/scsi_device/2:0:0:1/devicefound timeout at value 30new timeout value is 60. . .

# /sys/class/scsi_device/2:0:0:0/device# echo 60 > timeout# cat timeout60

set_target_timeout lpfc SCSI

: Emulex Emulex 30 60 udev

HP RHEL

Veritas Volume Manager

46

http://www.emulex.com

rpm -qa|grep device-mapper

: RHEL device-mapper-multipath

multipath Linux (dm)

multipathd

kpartx

udev sysfsutils udev /dev/ sysfsutils udev

:

# rpm -qa | grep udevudev-039-10.19.el4

# rpm -qa | grep sysfssysfsutils-devel-1.2.0-1sysfsutils-1.2.0-1

RHEL 5.4

# rpm -qa | grep udevudev-095-14.21.el5

# rpm -qa | grep sysfslibsysfs-2.0.0-6sysfsutils-2.0.0-6

/usr RHEL (/) libsysfs.so /usr/lib /lib

32 64

32

# cp /usr/lib/libsysfs.so.1.0.2 /lib/# ln -s /lib/libsysfs.so.1.0.2 /lib/libsysfs.so.1# ln -s /lib/libsysfs.so.1 /lib/libsysfs.so

47

64

# cp /usr/lib64/libsysfs.so.1.0.2 /lib64/# ln -s /lib64/libsysfs.so.1.0.2 /lib64/libsysfs.so.1# ln -s /lib64/libsysfs.so.1 /lib64/libsysfs.so

: /usr libsysfs.so.1 /usr libsysfs.so.1 /lib

: sysfsutils-xx libsysfs.so.1 /lib

/etc/multipath.conf /etc/multipath.conf HP 3PAR /etc/multipath.conf multipath.conf dm-multipath HP 3PAR HP 3PAR StoreServ Storage (SAN )

: SAN HP 3PAR StoreServ Storage (111 ) multipath.conf RHEL DM

1. /etc/multipath.conf

2. HP 3PAR

RHEL HP 3PAR HP 3PAR OS 3.1.1 /etc/multipath.conf

RHEL 4.xRHEL 5.5

cat /etc/multipath.confdefaults {}

devices {device {

vendor "3PARdata"product "VV"no_path_retry 18features "0"hardware_handler "0"path_grouping_policy multibusgetuid_callout "/sbin/scsi_id -g -u -s /block/%n"path_selector "round-robin 0"

48

rr_weight uniformrr_min_io 100path_checker turfailback immediatepolling_interval 10

}}

RHEL 5.6

# cat /etc/multipath.confdefaults {

polling_interval 10}

devices {device {

vendor "3PARdata"product "VV"no_path_retry 18features "0"hardware_handler "0"path_grouping_policy multibusgetuid_callout "/sbin/scsi_id -g -u -s /block/%n"path_selector "round-robin 0"rr_weight uniformrr_min_io 100path_checker turfailback immediate

}}

RHEL 6.1


polling_interval 10max_fds 8192

}

devices {device {

vendor "3PARdata"product "VV"no_path_retry 18features "0"hardware_handler "0"path_grouping_policy multibusgetuid_callout "/lib/udev/scsi_id --whitelisted

--device=/dev/%n"path_selector "round-robin 0"rr_weight uniformrr_min_io 1path_checker turfailback immediate

}}

49

RHEL 6.2



}

devices {device {

vendor "3PARdata"product "VV"no_path_retry 18features "0"hardware_handler "0"path_grouping_policy multibusgetuid_callout "/lib/udev/scsi_id --whitelisted

--device=/dev/%n"path_selector "round-robin 0"rr_weight uniformrr_min_io_rq 1path_checker turfailback immediate

}}

: RHEL HP 3PAR 3.1.1 HP 3PAR OS no_path_retry 18 12 polling_interval 10 5

1. multipath 2. multipathd rc 345

# chkconfig --list multipathdmultipathd 0:off 1:off 2:off 3:on 4:on 5:on 6:off

3. rc

# ls /etc/rc3.d/*multi*/etc/rc3.d/S13multipathd# ls /etc/rc5.d/*multi*/etc/rc5.d/S13multipathd

chkconfig

# chkconfig multipathd on

50

Veritas DMP / WebVERITAS Volume Manager Installation GuideVERITAS VolumeManager Administrator's GuideVeritas Storage Foundation and HighAvailability Software www.symantec.com ()

: Veritas Cluster V6.0.1 HP 3PAR OS 3.1.1 RHEL 6.x

: QLogic HBA Veritas DMP QLogic Non-FailoverDriver Veritas DMP (DM)

VERITAS Volume Manager Veritas

/lib/modules//veritas/vxvm/dmpaaa.ko/lib/modules//veritas/vxvm/dmpaa.ko /lib/modules//veritas/vxvm/dmpapf.ko/lib/modules//veritas/vxvm/dmpapg.ko/lib/modules//veritas/vxvm/dmpjbod.ko /lib/modules//veritas/vxvm/vxdmp.ko/lib/modules//veritas/vxvm/vxio.ko/lib/modules//veritas/vxvm/vxspec.ko/lib/modules//veritas/vxfs/fdd.ko/lib/modules//veritas/vxfs/vxfs.ko/lib/modules//veritas/vxfs/vxportal.ko

5.0mp3 Veritas Storage Foundation HP 3PARStoreServ Storage Veritas Array Support Library (ASL) RHEL HP 3PAR StoreServ Storage Veritas ASL

1. HP 3PAR Storage System Veritas ASL https://sort.symantec.com/asl/finder() 3PAR 5.0 SFHA ASL http://www.symantec.com/business/support/index?page=content&id=TECH61169 ()

: HP 3PAR Web ASL HP 3PAR

2. ASL Veritas vxconfigd vxinstall ASL vxdctl enable HP 3PAR

3. Veritas vxdmp HP 3PAR StoreServ Storage

4. Veritas vxdmp HP 3PAR StoreServ Storage Veritas

# vxddladm listsupport libname=libvx3par.soATTR_NAME ATTR_VALUE=======================================================================LIBNAME libvx3par.soVID 3PARdata

51

www.symantec.comhttps://sort.symantec.com/asl/finderhttp://www.symantec.com/business/support/index?page=content&id=TECH61169http://www.symantec.com/business/support/index?page=content&id=TECH61169

PID VVARRAY_TYPE A/AARRAY_NAME 3PARDATA

Veritas Storage Foundation 5.0mp3 HP 3PARStoreServ Storage ASL HP 3PAR StoreServStorage

# vxddladm listsupport libname=libvx3par.soATTR_NAME ATTR_VALUE=======================================================================LIBNAME libvx3par.soVID 3PARdataPID VVARRAY_TYPE A/AARRAY_NAME 3PARDATA

HP 3PAR StoreServ Storage 3PARDATA

# vxddladm addsupport all


# vxddladm listsupport libname=libvx3par.soATTR_NAME ATTR_VALUE=======================================================================LIBNAME libvx3par.soVID 3PARdataPID VVARRAY_TYPE A/AARRAY_NAME 3PARDATA

! ARRAY_NAME 3PARDATA

HP 3PAR Host Explorer HP 3PAR OS 2.3.1 OS 3.1.x RHEL Host Explorer HP 3PAR StoreServStorage HP BSC Web HP 3PAR Host Explorer Users Guidehttp://www.hp.com/go/bsc ()

52


6 iSCSI iSCSI iSCSI

Linux iSCSI HP 3PAR StoreServ Storage iSCSI

VLAN HP 3PAR StoreServ Storage iSCSI iSCSI VLAN iSCSI HP 3PAR StoreServ Storage iSCSI iSCSI iSCSI HP 3PARStoreServ Storage iSCSI iSCSI ping HP 3PAR StoreServ Storage iSCSI

: VLAN VLAN

HP 3PAR StoreServ Storage iSCSI iSCSI (NIC) (CNA)

iSCSI

RHEL 4 iSCSI iSCSI RHEL 4

RHEL 4 iSCSI iSCSI RHEL 4 OS CD RPM

RHEL 4 iSCSI RHEL 4 iSCSI 1. iSCSI chkconfig

# chkconfig --list | grep iscsiiscsi 0:off 1:off 2:off 3:off 4:off 5:off 6:off

2.

# runlevelN 5

iSCSI iSCSI 53

3. iSCSI iSCSI

# chkconfig --level 5 iscsi on# chkconfig --list | grep iscsiiscsi 0:off 1:off 2:on 3:on 4:on 5:on 6:off

4. /etc/iscsi/iscsid.conf HP 3PAR StoreServ Storage iSCSI IP 10.0.0.10 10.0.0.20 iSCSI

ConnFailTimeout=10DiscoveryAddress=10.0.0.10DiscoveryAddress=10.0.0.20

5. iSCSI

# /etc/init.d/iscsi reload/etc/init.d/iscsi reload

: : iSCSI /etc/init.d/iscsi reload /etc/iscsi/iscsid.conf iSCSI iscsi.conf

6. multipathd /etc/init.d/multipathd stop

# /etc/init.d/multipathd statusmultipathd is stopped

7. iscsi_sfnet

# lsmod | grep iscsi_sfnet

8. iscsi_sfnet

# lsmod | grep iscsi_sfnetiscsi_sfnet 96093 26scsi_transport_iscsi 14017 1 iscsi_sfnetscsi_mod 145297 7iscsi_sfnet,lpfc,libata,cciss,qla2xxx,scsi_transport_fc,sd_mod

RHEL 4 iSCSI /etc/multipath.conf HP 3PAR /etc/multipath.conf

54 iSCSI

multipath.conf dm-multipath HP 3PAR

: multipath.conf RHEL DM

: rpm (47 )


2. HP 3PAR


: RHEL HP 3PAR HP 3PAR OS

# cat /etc/multipath.confdefaults {}

devices {device {

vendor "3PARdata"product "VV"no_path_retry 12features "0"hardware_handler "0"path_grouping_policy multibuspath_selector "round-robin 0"rr_weight uniformrr_min_io 100path_checker turfailback immediatepolling_interval 5

}}

4. multipath 5. multipathd rc

345


6. rc


RHEL 4 iSCSI 55

chkconfig


RHEL 5 RHEL 6 iSCSI

RHEL 5 RHEL 6 iSCSI iSCSI RHEL iscsi-initiator-utils rpm iscsi-initiator-utils RHEL 2 RHEL CLI iscsi-initiator-utils GUI RHEL CLI iscsi-initiator-utils iscsiadm iSCSI open-iscsi iSCSI 1. IP 2. ID iSCSI 3. iSCSI

RHEL 56 iSCSI iSCSI iSCSI iSCSI I/O iSCSI /etc/iscsi/iscsid.conf

iSCSI replacement_timeout iSCSI /I/O replacement_timeout 120 replacement_timeout 1. /etc/iscsi/iscsid.conf

node.session.timeo.replacement_timeout = [replacement_timeout]

2. 5

node.session.timeo.replacement_timeout = 5

3. iSCSI iSCSI ping

node.conn[0].timeo.noop_out_interval = [replacement_timeout]

iSCSI iSCSI ping ping iSCSI ping

56 iSCSI

4. 10

node.conn[0].timeo.noop_out_interval = 10

5. iSCSI iSCSI /etc/iscsi/iscsid.conf iSCSI

node.startup = automaticnode.conn[0].startup = automatic

: node.conn[0] .startup iscsid

6. iSCSI chkconfig

# chkconfig --list | grep iscsiiscsi 0:off 1:off 2:off 3:off 4:off 5:off 6:offiscsid 0:off 1:off 2:off 3:on 4:on 5:on 6:off

7. 5

# chkconfig iscsi on# chkconfig --list|grep iscsiiscsi 0:off 1:off 2:on 3:on 4:on 5:on 6:offiscsid 0:off 1:off 2:off 3:on 4:on 5:on 6:off

8. /etc/iscsi/iscsid.conf node.session.cmds_max node.session.queue_depth

HP 3PAR Priority Optimization node.session.cmds_max node.session.queue_depth I/O Priority Optimization () HP 3PAR http://www.hp.com/go/bsc () HP 3PAR Priority Optimization

HP 3PAR StoreServ Storage HP 3PARPriority Optimization I/O (512) T F HP 3PAR 1G iSCSi node.session.cmds_max node.session.queue_depth

9.

CRC SCSI (CRC )PDU

RHEL 5 RHEL 6 iSCSI 57


() iSCSI PDU SCSI SCSI PDU CRC

: I/O

iSCSI /etc/iscsi/iscsid.conf

node.conn[0].iscsi.HeaderDigest = CRC32Cnode.conn[0].iscsi.DataDigest = CRC32C

: iSCSI

10. CHAP HP 3PAR StoreServ Storage CHAP (74 )

RHEL 5 RHEL 6 iSCSI BIOS IP OneCommand Manager GUI hbacmd iSCSI iSCSI Web OneCommandManager Command Line InterfaceVersion 6.1 User Manualhttp://www-dl.emulex.com/support/elx/r32/b16/docs/apps/ocm_cli_manual_elx.pdf ()

BIOS IP 1. BIOS IP

58 iSCSI

http://www-dl.emulex.com/support/elx/r32/b16/docs/apps/ocm_cli_manual_elx.pdf

3 IP

2. [Network Configuration] [Configure Static IP Address]


4 IP

3. [Static IP Address] [IP Address] [IP Address][Subnet Mask] [Default Gateway]

5 IP

60 iSCSI

4.

OneCommand Manager GUI OneCommand Manager GUI iSCSI

1. /usr/sbin/ocmanager/ocmanager & OneCommand Manager iSCSI

6 iSCSI

2. [Adapter information] [Personality] [Personality] [iSCSI]


7 [iSCSI]

3. 0:2:1

62 iSCSI

8

: HP 3PAR StoreServ showport -iscsi

4.


9

5. [Target Login...]

64 iSCSI

10

6.


11

7. [Target Sessions...]

66 iSCSI

12

HP 3PAR OS CLI createhost iscsi

# createhost -iscsi -persona 1 redhathost iqn.1990-07.com.emulex:28-92-4a-af-f5-61

LUN iSCSI IQN iqn.1990-07.com.emulex:28-92-4a-af-f5-61

( (67 ) )

hbacmd iSCSI hbacmd

iSCSI hbacmd


: iSCSI HP Web http://www8.hp.com/jp/ja/support.html ()

iSCSI hbacmd help

# hbacmd help

iSCSI

iSCSI IP

iSCSI

iSCSI

68 iSCSI

http://www8.hp.com/jp/ja/support.html

RHEL 5 RHEL 6 iSCSI /etc/multipath.conf HP 3PAR /etc/multipath.conf multipath.conf dm-multipath HP 3PAR

: multipath.conf RHEL DM



2. HP 3PAR

RHEL 5.0 RHEL 5.5

: RHEL HP 3PAR HP 3PAR OS

# cat /etc/multipath.conf

defaults {

}

devices {

device {

vendor "3PARdata"

product "VV"

no_path_retry 12

features "0"

hardware_handler "0"

path_grouping_policy multibus

path_selector "round-robin 0"

rr_weight uniform

rr_min_io 100

path_checker tur

failback immediate

polling_interval 5

}

}}

RHEL 5.6


polling_interval 5}

devices {device {

vendor "3PARdata"product "VV"no_path_retry 12features "0"hardware_handler "0"path_grouping_policy multibuspath_selector "round-robin 0"rr_weight uniformrr_min_io 100path_checker turfailback immediate

}}

RHEL 6.1


polling_interval 5}

70 iSCSI

devices {device {

vendor "3PARdata"product "VV"no_path_retry 12features "0"hardware_handler "0"path_grouping_policy multibuspath_selector "round-robin 0"rr_weight uniformrr_min_io 1path_checker turfailback immediate

}}

RHEL 6.2



}

devices {device {

vendor "3PARdata"product "VV"no_path_retry 12features "0"hardware_handler "0"path_grouping_policy multibuspath_selector "round-robin 0"rr_weight uniformrr_min_io_rq 1path_checker turfailback immediate

}}

3. multipathd 4. multipathd rc

345


5. rc



chkconfig


RHEL 5 RHEL 6 iSCSI RHEL iSCSI 1. open-iscsi

# /etc/init.d/iscsi startStarting iSCSI daemon:[ OK ][ OK ]

2. open-iSCSI chkconfig 5

# chkconfig --list | grep iscsiiscsi 0:off 1:off 2:off 3:off 4:off 5:on 6:off# chkconfig --list | grep iscsidiscsid 0:off 1:off 2:off 3:off 4:off 5:on 6:off

iSCSI

# chkconfig iscsi on# chkconfig iscsid on

iSCSI

# chkconfig --list iscsiiscsi 0:off 1:off 2:off 3:on 4:on 5:on 6:off# chkconfig --list iscsidiscsid 0:off 1:off 2:off 3:on 4:on 5:on 6:off

3. iscsi

# lsmod | grep iscsi

iscsi_tcp 56897 2libiscsi 59329 2 ib_iser,iscsi_tcpscsi_transport_iscsi 63569 4 ib_iser,iscsi_tcp,libiscsiscsi_mod 184057 10sg,ib_iser,iscsi_tcp,libiscsi,scsi_transport_iscsi,qla2xxx,lpfc,scsi_transport_fc,cciss,sd_mod

RHEL 5 RHEL 6 iscsiadm iSCSI

: ISCSI RHEL 5 RHEL 6 iSCSI (58 )

72 iSCSI

HP 3PAR StoreServ Storage iSCSI iscsiadm iSCSI 1. iscsiadm

iscsiadm -m discovery -t sendtargets -p :.

:

# iscsiadm -m discovery -t sendtargets -p 10.100.0.101:326010.100.0.101:3260,31 iqn.2000-05.com.3pardata:20310002ac000079

2. iscsiadm -m discovery :

# iscsiadm -m discovery10.100.0.101:3260 via sendtargets

3. iscsiadm -m node

# iscsiadm -m node10.100.1.101:3260,31 iqn.2000-05.com.3pardata:20320002ac000121

4. iSCSI iscsiadm -m node -T -p :-l.:

# iscsiadm -m node -T iqn.2000-05.com.3pardata:20310002ac000079 -p 10.100.0.101:3260 -lLogging in to [iface: default, target: iqn.2000-05.com.3pardata:20310002ac000079, portal: 10.100.0.101,3260]Login to [iface: default, target: iqn.2000-05.com.3pardata:20310002ac000079, portal: 10.100.0.101,3260]: successful

5. iscsiadm :

# iscsiadm -m node -T iqn.2000-05.com.3pardata:20310002ac000079 -p 10.100.0.101:326010.100.0.101:3260,31 iqn.2000-05.com.3pardata:20310002ac000079

6. iscsiadm -m session iSCSI

:

# iscsiadm -m sessiontcp: [1] 10.100.0.101:3260,31 iqn.2000-05.com.3pardata:20310002ac000079

RHEL iscsiadm RHEL iscsiadm (24 ) RHEL 5.4 open-iSCSI Linux iSCSI DBM

(/var/lib/iscsi/send_targets)


(/var/lib/iscsi/nodes)send_targets

send_targets/drw------- 2 root root 4096 Feb 26 16:51 10.102.2.131,3260drw------- 2 root root 4096 Feb 26 10:22 10.102.2.31,3260nodes/drw------- 3 root root 4096 Feb 26 10:22 iqn.2000-05.com.3pardata:20310002ac0000b1drw------- 3 root root 4096 Feb 26 10:58 iqn.2000-05.com.3pardata:21310002ac0000b1

send_targets iscsiadm send_targets nodes

: RHEL 5 iSCSI iface iSCSI NIC iscsiadm -m iface /var/lib/iscsi/ifaces iface RHEL 5 U4 open-iscsi

iSCSI CHAP 2 CHAP CHAP iSCSI HP 3PAR StoreServ Storage iSCSI () CHAP iSCSI iSCSI CHAP HP 3PAR StoreServ Storage iSCSI iSCSI (21 )

HP 3PAR StoreServ Storage CHAP CHAP iSCSI HP 3PAR StoreServ Storage HP 3PAR OS CLI sethost initchap CHAP HP 3PAR OS 3.1.x OS 2.3.x

# showhostId Name Persona ----------WWN/iSCSI_Name----------- Port0 redhatlinux Generic iqn.1994-05.com.redhat:a3df53b0a32d ---

HP 3PAR OS 2.2.x

# showhostId Name -----------WWN/iSCSI_Name------------ Port0 linux iqn.1994-05.com.redhat:a3df53b0a32d --

iqn.1994-05.com.redhat:a3df53b0a32d

CHAP host_secret0 CHAP 12

CHAP

# sethost initchap -f host_secret0 redhatlinux

74 iSCSI

CHAP

# showhost -chapId Name -Initiator_CHAP_Name- -Target_CHAP_Name-0 redhatlinux redhatlinux --

RHEL 5 RHEL 6 CHAP RHEL 5 RHEL 6 CHAP 1. iSCSI /etc/iscsi/iscsid.confCHAP

#To enable CHAP authentication set node.session.auth.authmethod#to CHAP. The default is None.node.session.auth.authmethod = CHAP

2. CHAP /etc/iscsi/iscsid.conf

#To set a discovery session CHAP username and password for the initiator#authentication by the target(s), uncomment the following lines:discovery.sendtargets.auth.username = redhatlinuxdiscovery.sendtargets.auth.password = host_secret0#To set a CHAP username and password for initiator#authentication by the target(s), uncomment the following lines:node.session.auth.username = redhatlinuxnode.session.auth.password = host_secret0

: OutgoingUsername OutgoingPasswordHP 3PAR StoreServ Storage CHAP

3. iSCSI (96 )

iSCSI

a. iSCSI


b. iSCSI

# iscsiadm -m node -o delete -T iqn.2000-05.com.3pardata:20310002ac000079 -p10.100.0.101,3260

c. SendTarget iSCSI


iSCSI CHAP 75

d. iSCSI

# /etc/init.d/iscsid stopStopping iSCSI daemon:# /etc/init.d/iscsid startTurning off network shutdown. Starting iSCSI daemon: [ OK ][ OK ]

e. RHEL 5 RHEL 6 iscsiadm iSCSI (72 ) iSCSI iSCSI

RHEL 4 CHAP RHEL 4 CHAP 1. iSCSI /etc/iscsi/iscsid.conf CHAP

DiscoveryAddress=10.0.0.10DiscoveryAddress=10.0.0.20OutgoingUsername=redhatlinuxOutgoingPassword=host_secret0

: OutgoingUsername OutgoingPassword DiscoveryAddress

: OutgoingUsername OutgoingPasswordHP 3PAR StoreServ Storage CHAP

2. iscsid /etc/init.d/iscsi status

# /etc/init.d/iscsi statusiscsid (pid 30532 30529) is running... (RedHat 4)Checking for service iSCSI driver is loaded

HP 3PAR StoreServ Storage CHAP CHAP () HP 3PAR StoreServ Storage CHAP HP 3PAR OS CLI sethost initchap sethosttargetchap 1. HP 3PAR StoreServ Storage CHAP host_secret0 CHAP target_secret0 HP 3PAR OS 3.1.x OS 2.3.x

# showhostId Name Persona ----------WWN/iSCSI_Name----------- Port0 redhatlinux Generic iqn.1994-05.com.redhat:a3df53b0a32d ---

76 iSCSI

HP 3PAR OS 2.2.x showhost iSCSI HP 3PARStoreServ Storage

# showhostId Name -----------WWN/iSCSI_Name------------ Port0 linux iqn.1994-05.com.redhat:a3df53b0a32d --

iqn.1994-05.com.redhat:a3df53b0a32d --

: CHAP host_secret0 CHAP 12

2. CHAP

# sethost initchap -f host_secret0 redhatlinux

3. CHAP

# sethost targetchap -f target_secret0 redhatlinux

4. CHAP

# showhost -chapId Name -Initiator_CHAP_Name- -Target_CHAP_Name-0 redhatlinux redhatlinux S121

RHEL 5 RHEL 6 CHAP RHEL 5 RHEL 6 CHAP iSCSI /etc/iscsi/iscsid.conf CHAP

: HP 3PAR StoreServ Storage iSCSI IP 2 DiscoveryAddress 1 CHAP (OutgoingUsername OutgoingPassword) 1 CHAP (IncomingUsername IncomingPassword)

1. CHAP

# To enable CHAP authentication set node.session.auth.authmethod# to CHAP. The default is None.node.session.auth.authmethod = CHAP

# To set a discovery session CHAP username and password for the initiator# authentication by the target(s), uncomment the following lines:discovery.sendtargets.auth.username = redhatlinuxdiscovery.sendtargets.auth.password = host_secret0

# To set a CHAP username and password for initiator# authentication by the target(s), uncomment the following lines:node.session.auth.username = redhatlinuxnode.session.auth.password = host_secret0

iSCSI CHAP 77

2. CHAP

#To set a discovery session CHAP username and password for target(s)# authentication by the initiator, uncomment the following lines:discovery.sendtargets.auth.username_in = S121discovery.sendtargets.auth.password_in = target_secret0

# To set a CHAP username and password for target(s)# authentication by the initiator, uncomment the following lines:node.session.auth.username_in = S121node.session.auth.password_in = target_secret0

: S121 CHAP showhost -chap HP 3PAR StoreServ Storage

: OutgoingUsername IncomingUsername OutgoingPassword IncomingPassword HP 3PAR StoreServ Storage CHAP CHAP

3. iSCSI (96 )

iSCSI

a. iSCSI


b. iSCSI

# iscsiadm -m node -o delete -T iqn.2000-05.com.3pardata:20310002ac000079 -p10.100.0.101,3260

c. SendTarget iSCSI


d. iSCSI

# /etc/init.d/iscsid stopStopping iSCSI daemon:# /etc/init.d/iscsid startTurning off network shutdown. Starting iSCSI daemon: [ OK ][ OK ]

e. iSCSI

/var/lib/iscsi/send_targets//var/lib/iscsi/nodes/

78 iSCSI

:

# ls -l /var/lib/iscsi/send_targets/*# ls -l /var/lib/iscsi/nodes/*# rm -rf /var/lib/iscsi/send_targets/*# rm -rf /var/lib/iscsi/nodes/*

f. RHEL 5 RHEL 6 iscsiadm iSCSI (72 ) iSCSI iSCSI

RHEL 4 CHAP RHEL 4 CHAP 1. iSCSI /etc/iscsi/iscsid.conf CHAP

DiscoveryAddress=10.0.0.10DiscoveryAddress=10.0.0.20

OutgoingUsername=redhatlinuxOutgoingPassword=host_secret0

DiscoveryAddress=10.0.0.10DiscoveryAddress=10.0.0.20

IncomingUsername=S4121IncomingPassword=target_secret0

: HP 3PAR StoreServ Storage iSCSI IP 2 DiscoveryAddress 1 CHAP (OutgoingUsername OutgoingPassword) 1 CHAP (IncomingUsername IncomingPassword)

: OutgoingUsername IncomingUsername OutgoingPassword IncomingPassword HP 3PAR StoreServ Storage CHAP CHAP

DiscoveryAddress DiscoveryAddress

2. iscsid daemon /etc/init.d/iscsi

3. iscsid /etc/init.d/iscsi status

# /etc/init.d/iscsi statusiscsid (pid 30532 30529) is running... (RedHat 4)Checking for service iSCSI iSCSI driver is loaded

iSCSI CHAP 79

: RHEL iSCSI (583218) RHEL

http://rhn.redhat.com/errata/RHBA-2011-0075.html ()

Internet Storage Name Server Internet Storage Name Server (ISNS) IP

: ISNS DHCP iSCSI

Microsoft iSNS iSCSI iSCSI MicrosoftiSNS Windows 2008 iSNS iSNS

iSNS

1. [][][iSNS ]+[]2. []3. [] [OK]

iSNS iSCSI

HP 3PAR StoreServ Storage HP 3PAR StoreServ Storage 1. showport -iscsi iSCSI iSNS :

2. iSNS IP

# controliscsiport isns

:

# controliscsiport isns 10.107.66.11 0:3:2# controliscsiport isns 10.107.66.11 1:3:2

80 iSCSI

http://rhn.redhat.com/errata/RHBA-2011-0075.html

3. iSNS :

iSNS (RHEL ) yum isns-utils

# yum --nogpgcheck install isns-utils

# chkconfig isnsd on

# service isnsd start

iSNS

# iscsiadm -m iface -o new -I isns_iface

TCP/IP

# iscsiadm -m iface -o update -I isns_iface -n iface.transport_name -v tcp

iSNS :

# iscsiadm -m discoverydb -t isns -p : -o new# iscsiadm -m discoverydb -t isns -p 10.107.66.11:3205 -o update -ndiscovery.isns.use_discoveryd -v Yes

/var/lib/iscsi/isns/,/isns_config 30

# vi /var/lib/iscsi/isns/10.107.66.11,3205/isns_config

iSCSI

# service iscsid restart

Internet Storage Name Server 81

iSNS

# service isnsd restart

:

82 iSCSI

7 FCoE Linux Linux FCoE HP 3PAR StoreServ Storage HP 3PAR StoreServ Storage FCoE

Linux Linux HP SPOCK Web http://www.hp.com/storage/spock ()

HBA BIOS http://www.hp.com/go/spp ()

HBA http://www8.hp.com/jp/ja/support.html ()

Emulex OneCommand Manager (/usr/sbin/ocmanager/hbacmd) QLogicQConvergeConsole Manager(/opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli) FCoE Web

FCoE RHEL (FCoE ) HP 3PAR StoreServ Storage (FCoE ) FCoE

: FCoE VLAN VLAN

BIOS FCoE 1. BIOS FCoE [F9]

[Setup]

Linux 83

http://www.hp.com/storage/spockhttp://www.hp.com/go/spphttp://www8.hp.com/jp/ja/support.htmlhttp://www8.hp.com/jp/ja/support.html

13 FCoE

2. [System Options] [NIC Personality Options]

14 NIC Personality Options

3. [PCI Slot 2] [Port 1] [Port 2] [FCoE]

84 FCoE

15 PCI

4. [PCI Slot 2 Port 1] [Port 2] [FCoE]

16 FCoE PCI Slot 1 Slot 2

5. BIOS

BIOS FCoE 85

17 BIOS

86 FCoE

8 RHEL HP 3PAR StoreServ Storage

(VV) RHEL

HP 3PAR Command Line Interface Administrators Manual HP 3PAR OS HP 3PAR Command Line InterfaceReference HP Web http://www.hp.com/go/3par/ () [HP Support & Drivers][Manuals] http://www.hp.com/jp/manual/ ()

HP 3PAR StoreServ Storage RHEL HP 3PAR StoreServ Storage

1 CPG (Common Provisioning Group) 1 CPG 1 CPG

HP 3PAR Management Console 1.

[Actions][Provisioning][Virtual Volume][Create Virtual Volume]2. [Create Virtual Volume] 3. [Allocation]

[Fully Provisioned]

[Thinly Provisioned]

HP 3PAR OS CLI HP 3PAR OS CLI

# createvv [options] [.] [g|G|t|T]

# createvv -cnt 5 TESTLUNS 5G

: HP 3PAR Thin Provisioning

HP 3PAR StoreServ Storage HP 3PAR OS HP 3PAR Management Console HelpHP 3PAR Command LineInterface ReferenceHP BSC Web

HP 3PAR StoreServ Storage 87

http://www.hp.com/go/3par/http://www.hp.com/go/3par/


: HP 3PAR OS

(TPVV)

HP 3PAR StoreServ Storage Concepts Guide

HP 3PAR Command Line Interface Administrators Manual

HP 3PAR Command Line Interface ReferenceHP BSC Web http://www.hp.com/go/bsc ()

LUN LUN LUN LUN (VLUN) LUN

createvlun [cnt]

[cnt] VV 1999 1

LUN

LUN

.int LUN (.int) 1

(10 ) (12 ) .

:

# createvlun cnt 5 TESTLUNS.0 0 hostname/hostdefinition

VLUN showvlun

# showvlun

Active VLUNsLun VVName HostName --------Host_WWN/iSCSI_Name-------- Port Type

0 TESTLUNS.0 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 0:3:1 host1 TESTLUNS.1 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 0:3:1 host2 TESTLUNS.2 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 0:3:1 host3 TESTLUNS.3 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 0:3:1 host4 TESTLUNS.4 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 0:3:1 host5 TESTLUNS.5 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 0:3:1 host6 TESTLUNS.6 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 0:3:1 host7 TESTLUNS.7 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 0:3:1 host

88 RHEL

http://www.hp.com/go/bschttp://www.hp.com/go/bsc

8 TESTLUNS.8 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 0:3:1 host9 TESTLUNS.9 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 0:3:1 host0 TESTLUNS.0 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 1:3:1 host1 TESTLUNS.1 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 1:3:1 host2 TESTLUNS.2 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 1:3:1 host3 TESTLUNS.3 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 1:3:1 host4 TESTLUNS.4 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 1:3:1 host5 TESTLUNS.5 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 1:3:1 host6 TESTLUNS.6 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 1:3:1 host7 TESTLUNS.7 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 1:3:1 host8 TESTLUNS.8 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 1:3:1 host9 TESTLUNS.9 sqa-dl380g5-05 iqn.1994-05.com.redhat:33853dd5ab2e 1:3:1 host

----------------------------------------------------------------------------20 total

VLUN TemplatesLun VVName HostName -Host_WWN/iSCSI_Name- Port Type

0 TESTLUNS.0 sqa-dl380g5-05 ---------------- --- host1 TESTLUNS.1 sqa-dl380g5-05 ---------------- --- host2 TESTLUNS.2 sqa-dl380g5-05 ---------------- --- host3 TESTLUNS.3 sqa-dl380g5-05 ---------------- --- host4 TESTLUNS.4 sqa-dl380g5-05 ---------------- --- host5 TESTLUNS.5 sqa-dl380g5-05 ---------------- --- host6 TESTLUNS.6 sqa-dl380g5-05 ---------------- --- host7 TESTLUNS.7 sqa-dl380g5-05 ---------------- --- host8 TESTLUNS.8 sqa-dl380g5-05 ---------------- --- host9 TESTLUNS.9 sqa-dl380g5-05 ---------------- --- host

-------------------------------------------------------------10 total

HP 3PAR Command Line Interface Administrators Manual(VV) LUN (VLUN)

LUN (LUN ) LUN (: 0573 )

HP 3PAR StoreServ Storage VLUN 065535 LUN

RHEL LUN HP 3PAROS 2.3.x 3.1.x 16TB RHEL 16 TB LUN RHEL RHEL 2 TB

Emulex HBA LUN

echo

echo scsi add sys device echo

LUN RHEL LUN

1: sysfs scan

89

2: adding single devices 1

1 - sysfs scanLUN (88 ) createvlun VLUNecho sysfs cat/proc/scsi/scsi lsscsi -g sginfo -l option

# echo - >

: /sys/class/scsi_host/host2 0 (target2:0:0) LUN 1 echo

# echo "- 0 1" > /sys/class/scsi_host/host2/scan

kernel: Vendor: 3PARdata Model: VV Rev: 0000kernel: Type: Direct-Access ANSI SCSI revision: 03kernel: SCSI device sdv: 524288 512-byte hdwr sectors (268 MB)kernel: SCSI device sdv: drive cache: write backkernel: sdv: unknown partition tablekernel: Attached scsi disk sdv at scsi2, channel 0, id 0, lun 1kernel: Attached scsi generic sg22 at scsi2, channel 0, id 0, lun 1, type 0scsi.agent[12915]: disk at /devices/pci0000:00/0000:00:02.0/0000:01:00.2/0000:03:0b.0/0000:04:04.0/host2/target2:0:0/2:0:0:1

lpfc LUN

# echo "- - -" > /sys/class/scsi_host/host2/scan

lpfc LUN

# /usr/bin/rescan-scsi-bus.sh -r --nooptscan

# echo 1 > /sys/class/scsi_device/2:0:0:1/device/rescan

# multipathd -kmultipathd> resize map 350002ac000350102okmultipathd> exit

90 RHEL

2 - echo scsi add LUN

# echo "scsi add-single-device 0 1 2 3" >/proc/scsi/scsi

0

1

2 ID

3 LUN (SCSI )

# echo "scsi add-single-device 2 0 0 14" > /proc/scsi/scsi

SCSI OS LUN /var/log/messages

kernel: Vendor: 3PARdata Model: VV Rev: 0000kernel: Type: Direct-Access ANSI SCSI revision: 03kernel: SCSI device sdac: 524288 512-byte hdwr sectors (268 MB)kernel: SCSI device sdac: drive cache: write backkernel: sdac: unknown partition tablekernel: Attached scsi disk sdac at scsi2, channel 0, id 0, lun 14kernel: Attached scsi generic sg29 at scsi2, channel 0, id 0, lun 14, type 0Dec 12 14:08:50 sqa-dell2850-01 scsi.agent[14234]: disk at /devices/pci0000:00/0000:00:02.0/0000:01:00.2/0000:03:0b.0/0000:04:04.0/host2/target2:0:0/2:0:0:14

: echo LUN lpfc HBA SCSI

Emulex HBA RHEL /proc/scsi/scsi LUN 0 RHEL 8 (4 HP 3PAR StoreServ Storage 2 Emulex HBA ) HP 3PAR StoreServ Storage

# cat /proc/scsi/scsi[root@sqa-dl380g5-05 ~]# cat /proc/scsi/scsiAttached devices:Host: scsi4 Channel: 00 Id: 00 Lun: 00

Vendor: 3PARdata Model: VV Rev: 3110Type: Direct-Access ANSI SCSI revision: 05

Host: scsi5 Channel: 00 Id: 00 Lun: 00Vendor: 3PARdata Model: VV Rev: 3110Type: Direct-Access ANSI SCSI revision: 05

scsi4 scsi5 HBA (/sys/class/scsi_host/host4 /sys/class/scsi_host/host5) RedHat 5 6 Isscsi -g

Emulex HBA 91

QLogic HBA LUN echo QLogic scsi-qlascan echo scsi add

scsi-qlascan

QLogic scsi-qlascan echo

echo scsi add sys device tree echo

LUN RHEL LUN

1: echo sysfs

2: add single device 1

1 - echo sysfs LUN (88 ) createvlun VLUNQLogic scsi-qlascan

: OS QLogic scsi -qlascan echo

# echo " - " >

1. scsi-qlascan

# echo "scsi-qlascan" > /proc/scsi/qla2xxx/

: scsi-qlascan QLogic QLogic Web Linux QLogic Web QLogic

0 qla2xxx HBA

# echo "scsi-qlascan" > /proc/scsi/qla2xxx/0

2. HBA QLogic HP 3PAR StoreServ Storage

: qla2xxx QLogic Linux QLogic Web

92 RHEL

:

# cat /proc/scsi/qla2xxx/0QLogic PCI to Fibre Channel Host Adapter for QLA2462:

Firmware version 4.06.03 [IP] [84XX] , Driver version 8.02.23BIOS version 1.29FCODE version 1.27EFI version 1.09Flash FW version 4.00.30 0082ISP: ISP2422, Serial# RFC0823R29292Request Queue = 0x12a100000, Response Queue = 0x12a690000Request Queue count = 4096, Response Queue count = 512Total number of active commands = 0Total number of interrupts = 12368

Device queue depth = 0x20Number of free request entries = 282Number of mailbox timeouts = 0Number of ISP aborts = 0Number of loop resyncs = 0Number of retries for empty slots = 0Number of reqs in pending_q= 0, retry_q= 0, done_q= 0, scsi_retry_q= 0Host adapter:loop state = , flags = 0x5a43Dpc flags = 0x0MBX flags = 0x0Link down Timeout = 030Port down retry = 001Login retry count = 008Commands retried with dropped frame(s) = 0Product ID = 0000 0000 0000 0000

SCSI Device Information:scsi-qla0-adapter-node=2000001b321a0c63;scsi-qla0-adapter-port=2100001b321a0c63;scsi-qla0-target-0=20410002ac000031;scsi-qla0-target-1=20510002ac000031;scsi-qla0-target-2=21410002ac000031;scsi-qla0-target-4=21510002ac000031;

FC Port Information:scsi-qla0-port-0=2ff70002ac000031:20410002ac000031:090800:81;scsi-qla0-port-1=2ff70002ac000031:20510002ac000031:050100:82;scsi-qla0-port-

3PAR Red Hat Enterprise Linux およびOracle Linux 実装ガ …€¦ · · 2014-04-031概要 この実装ガイドでは、Red Hat Enterprise Linux (RHEL) 4、RHEL 5、RHEL 6、および

Documents

3PAR Red Hat Enterprise Linux およびOracle Linux 実装ガ …€¦ · · 2014-04-031概要この実装ガイドでは、Red Hat Enterprise Linux (RHEL) 4、RHEL 5、RHEL 6、および