SYMPOSIUM 2015 DATA CENTER SYMPOSIUM 2015 DATA CENTER Managing and Monitoring Storage Networks in 2015 – Infrastructure Matters Rob Peloquin, Brocade
SYMPOSIUM2015
DATA CENTERSYMPOSIUM
2015
DATA CENTER
Managing and Monitoring
Storage Networks in 2015 – Infrastructure Matters
Rob Peloquin, Brocade
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Legal Disclaimer
• All or some of the products detailed in this presentation may still be under development and certain specifications, including but not limited to, release dates, prices, and product features, September change. The products September not function as intended and a production version of the products September never be released. Even if a production version is released, it may be materially different from the pre-release version discussed in this presentation.
• NOTHING IN THIS PRESENTATION SHALL BE DEEMED TO CREATE A WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, STATUTORY OR OTHERWISE, INCLUDING BUT NOT LIMITED TO, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT OF THIRD-PARTY RIGHTS WITH RESPECT TO ANY PRODUCTS AND SERVICES REFERENCED HEREIN.
• Brocade, the B-wing symbol, BigIron, DCX, Fabric OS, FastIron, IronView, NetIron, SAN Health, ServerIron, and TurboIron are registered trademarks, and Brocade Assurance, DCFM, Extraordinary Networks, and Brocade NET Health are trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries. Other brands, products, or service names mentioned are or may be trademarks or service marks of their respective owners.
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Fibre Channel SAN SwitchMarket Share by Port Shipment—Q1’15
Source: Dell’Oro Group, 1Q15,”SAN Switch Share (Fixed + Modular) Total Modular Switches
Fixed Switches
BRCD69.5%
Cisco 18.7%
QLogic8.4%
Others 1.4%
BRCD
76.4%
QLogic11.4%
Cisco 10.3%
Others 1.9%
BRCD
61.0%
Cisco 39.0%
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Brocade Data Center Fabric SolutionsBest-in-class FC solutions
SAN Extension
Solutions
Brocade DCX
Backbones
48, 64, 80 ports
Brocade 5300
8, 16, 24 ports
Brocade 300
8 Gbps
Brocade VA40-FC
24, 32, 40 portsBrocade 5100
24, 32, 40 ports
Brocade DCX 8510
Backbones
Brocade 6510
24, 36, 48 ports
16 Gbps
Brocade 6520
48, 72, 96 ports
Brocade 7840
Brocade 7800
FX8-24
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Fibre Channel Futures -> GEN 6
• Gen 6 Fibre Channel standards
• FCIA 4 Minute Video on GEN6 Fibre Channel:
• https://www.youtube.com/watch?v=aksYjopxxWQ&list=PLTeNsFQA8JYc6xxqZxoKhKiZ2MMMjPU5u&index=13
• Gen 6 Fibre Channel is now the official name of next generation Fibre Channel
• Pre-production products to OEM’s in 2016
• Featured Gen 6 Enhancements
• Forward Error Correction (FEC) required to bolster reliability and availability
• Parallel Fibre Channel (128Gb)
• FC-EE improves the data center green footprint
32Gb and 128Gb Fibre Channel are Coming
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Brocade DCX 8510-8
• Up to 384 ports w/standard SFPs, 512 ports using QSFPs
• Up to 32 optical Inter-Chassis Link (ICL) ports
• 8.2 Tbps total chassis bandwidth
• 6.1 Tbps universal ports
• 2.1 Tbps ICL bandwidth
• 512 Gbps bandwidth per slot
• 12-slot card cage w/ eight port blades
• 32, 48, 64 Port blades
• Two Control Processor (CP) blades
• Two Core (CR) switching blades
• Eight slots for port and specialty blades
Large enterprises
© 2015 Brocade Communications Systems, Inc.
2 CP Blades2 CR Blades
32 ICL ports
8 slots
14U
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
• Up to 192 ports w/standard SFPs, 256 ports using QSFPs
• Up to 16 optical Inter-Chassis Link (ICL) ports
• 4.1 Tbps total chassis bandwidth
• 3.1 Tbps universal ports
• 1.0 Tbps ICL bandwidth
• 512 Gbps bandwidth per slot
• 8-slot horizontal card cage
• Two control processor (CP8) blades
• Two core routing (CR16-4) blades
• Four slots for port and specialty blades
9U
Brocade DCX 8510-4 Midsize enterprises
© 2015 Brocade Communications Systems, Inc.
9U height includes 8U chassis height plus 1U bottom exhaust shelf
2 CP Blades2 CR Blades
16 ICL ports
4 slots
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
CABLExpress ICL Cable Options
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Brocade Director Blade OptionsFor both the 4 and 8-slot Directors
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
GEN5 48-Port 16 Gbps Blade
• 2 × Condor3 16 Gbps ASICs
• 48 universal Fibre Channel/FICON ports (E, F, D, M, and EX)
• Front end ports can operate at 2, 4, 8, 10, and 16 Gbps
• 48 front-end ports and 32 back-end ports -1.5:1 oversubscription, no over subscription with local switching
• Dynamic power monitoring capability
Model #:FC16-48
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
CABLExpress –Cabling the 384-Port (8) 48-Port Line Card Option
GEN5 48-Port 16 Gbps Blade
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
GEN5 32-Port 16 Gbps Blade
• 2 × Condor3 16 Gbps ASICs
• 32 universal Fibre Channel/FICON ports (E, F, D, M, and EX)
• Front end ports can operate at 2, 4, 8, 10, and 16 Gbps
• 32 front-end ports and 32 back-end ports, no oversubscription
• Dynamic power monitoring capability
FC16-32
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
CABLExpress –Cabling the 256-Port (8) 32-Port Line Card option
© 2015 Brocade Communications Systems, Inc.
GEN5 32-Port 16 Gbps Blade
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
GEN5 FC16-64 Blade Overview
© 2014 Brocade Communications Systems, Inc.
16 x 4/8/16Gb QSFPs
Port 63 LED
Power On LED
Blade Status LED
Ports 63..0(QSFP ports 15..0)
• Ports are numbered from 0 through 63 from bottom to top
• Trunking port groups are 0-7, 8-15, 16-23, 24-31, 32-39, 40-47, 48-55, and 56-63 as marked on faceplate
Port 0 LED
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
GEN5 64-Port Blade - DCX 8510FC16-64 key product comparison versus FC8-64
64 x2/4/8 Gbps
mSFP+ ports
FC8-64 Port Blade
FC8-64 FC16-64
High density port blade
No Simplified cable deployment
No 16Gbps port speeds
No Condor 3 architecture
No Gen 5 capable
NoAdvanced Fabric Vision
features
NoLowest power per Gb
throughput
16 x 4/8/16 Gbps
QSFP ports
FC16-64 Port Blade
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
DCX 8510 w/ GEN 5 64 Port BladeFC16-64 extends lead in energy efficiency by 49% over FC8-64
© 2014 Brocade Communications Systems, Inc.Proprietary Information
DCX 8510 w/ FC16-64 blades*
CP8 blades (2) 80
CR16-8 (2) 480
Fans (3) 270
FC16-64 blades (8) 1072Total 1902
Watts/Gb .23
* Only two power supplies are required to power a fully loaded DCX-8510 chassis
The industry’s lowest power consumption!
DCX 8510 w/ FC8-64 blades
CP8 blades (2) 80
CR16-8 (2) 480
Fans (3) 270
FC8-64 blades (8) 1011.2
Total 1841.2
Watts/Gb .45
Power Savings vs FC8-64 (per Gb)
- 49%
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
FC16-64 Simplified Cable ManagementEasier to deploy, fewer required cables, industry standard
• Utilizes QSFP ports to reduce port density from 64 to 16
• Reduces cabling requirements by 75%
• Simplifies cable deployment, troubleshooting and management
• QSFP optic modules support the following cable options• MTP (MPO) to LC
breakout cables• MTP to MTP patch panel
MTP to Patch Panel
MTP MTP
MTP to LC Cables
MTP
LC
LC
LC
LC
Cable Examples Results!
Fully populatedDCX 8510 with
FC8-64
Fully populatedDCX 8510 with
FC16-64
• 8 blades with 64 ports each requires 512 cables
• 8 blades with 16 ports each requires only 128 cables
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Brocade ReliabilityWorking to keep applications as highly available as possible
© 2015 Brocade Communications Systems, Inc.
Brocade GEN4/5 Cisco GEN4 blade Cisco GEN5 blade
“Brocade switches and directors have a lower hardware replacement rate than all other SAN vendors.” - OEM Partner
Brocade Sample size Downtime per
unit (minutes)
Availability
rate
All products 26,593 0.53 99.99984%
Brocade Rated #1 in Quality
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
BNA – Brocade Network AdvisorWorld-class configuration, monitoring and troubleshooting tool
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
BNA Main Dashboard for Network Health Monitoring - Key SAN Management Features
• Out-of-box collectors
• Four widget types for viewing data
• Ability to define custom collectors
• Customizable Dashboard
• Drill-down into detailed view
• Export dashboard/widget views
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
BNA Dashboards Out-of-Box Performance Dashboards
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
BNA Dashboards Current and Historical Time Scopes
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Monitoring and Alerting Policy Suite (MAPS)
Provides the ability to define and monitor policy
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
MAPS Configuration Dialog
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Cabling Checks
• Ensure you’re ready for 16Gbps and more …
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Infrastructure Issues – the ‘Real’ World
• Examples of actual customer outputs
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Bad Cabling/Connectors/SFPs Pure Storage Array – Performance Issues - the below is from a SAN Health report output:
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Bad Cabling/Connectors/SFPs
Lost BB Credits on a link - the below is from a SAN Health report output:
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Bad Cabling/Connectors/SFPs
‘sfpshow x/y –f’ – partial output from a Director …
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Bad Cabling/Connectors/SFPsC3TXTOs – (discarded C3 frames due to timeouts) – BNA and MAPS
Network Advisor Version: 12.1.6Network Advisor Build: 7Network Advisor Server Address: XX.YY.ZZ.220Date: Mon Aug 03 09:04:23 EDT 2015Event: Port 10/11, Condition=ALL_OTHER_F_PORTS(C3TXTO/min>3), Current Value:[C3TXTO, 1193 Timeouts], RuleName=defALL_OTHER_F_PORTSC3TXTO_3, Dashboard Category=Port Health.Severity: Error
RuleName=defALL_TARGET_PORTSC3TXTO_5, Condition=ALL_TARGET_PORTS(C3TXTO/min>5), Obj:Port 23 [C3TXTO,108Timeouts].
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Bad Cabling/Connectors/SFPs
C3TXTOs – (discarded C3 frames due to timeouts) – Switch CLI2015/08/11-20:02:27, [MAPS-1002], 33851, SLOT 6 | FID 128, ERROR, brocade-dcx-a, Port 2/8, Condition=ALL_TARGET_PORTS(C3TXTO/min>3), Current Value:[C3TXTO,458 Timeouts], RuleName=defALL_TARGET_PORTSC3TXTO_3, Dashboard Category=Port Health.
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Bad Cabling/Connectors/SFPsInvalid Transmission Words – MAPS Alert
Invariably, these conditions don’t tend to get better ;)
Apr 25 09:18:28 Port 11/18 Condition: ALL_HOST_PORTS(ITW/min>21) Current Value: [ITW,129 ITWs], Rule name: defALL_HOST_PORTSITW_21, Dashboard Category: Port Health
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Bad Cabling/Connectors/SFPsLink Failures (LFs) – MAPS rule violations
Subject: Port 3/0 triggered LF rule - defNON_E_F_PORTSLF_3 at Jul 19 19:17:48Switch Time: Jul 19 19:17:48Affected Entity: Port 3/0Rule Name:defNON_E_F_PORTSLF_3Condition:NON_E_F_PORTS(LF/min>3)Current Value:[LF,5]Dashboard Category: Port HealthSwitch Name:core2-fabaSwitch WWN:10:00:00:05:33:1b:xx:yySwitch IP:10.30.xx.yyFabric Name:uninitializedVFID:128
Regarding: Apr 29 18:04:35 Port 7/30 Condition: ALL_OTHER_F_PORTS(LF/min>3) Current Value: [LF,7], Rule name: defALL_OTHER_F_PORTSLF_3, Dashboard Category: Port Health
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Bad Cabling/Connectors/SFPsSignal Sync Loss – MAPS rule violations
© 2015 Brocade Communications Systems, Inc.
Jun 19 19:25:42 Port 2/45 Condition: NON_E_F_PORTS(LOSS_SYNC/min>3) Current Value: [LOSS_SYNC,5 SyncLoss], Rule name: defNON_E_F_PORTSLOSS_SYNC_3, Dashboard Category: Port Health
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Bad Cabling/Connectors/SFPsBNA MAPS output – captured from a screenshot:
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Bad Cabling/Connectors/SFPs
BNA MAPS output – captured from a screenshot:
© 2015 Brocade Communications Systems, Inc.
DATA CENTER SYMPOSIUM 2015DATA CENTER SYMPOSIUM 2015
Infrastructure Matters …I see it every week …
© 2015 Brocade Communicatios Systems, Inc.