Mesh Control

B0700AZREV L

I/A Series®SystemThe MESH Control Network Architecture Guide

October 15, 2009

Invensys, Foxboro, FoxView, I/A Series, and the IPS logo are trademarks of Invensys plc, its subsidiaries and affiliates.All other brand names may be trademarks of their respective owners.
Copyright 2004-2009 Invensys Systems, Inc.All rights reserved

SOFTWARE LICENSE AND COPYRIGHT INFORMATION

Before using the Invensys Systems, Inc. supplied software supported by this documentation, you should read and understand the following information concerning copyrighted software.

1. The license provisions in the software license for your system govern your obligations and usage rights to the software described in this documentation. If any portion of those license provisions is violated, Invensys Systems, Inc. will no longer provide you with support services and assumes no further responsibilities for your system or its operation.

2. All software issued by Invensys Systems, Inc. and copies of the software that you are specifically permitted to make, are protected in accordance with Federal copyright laws. It is illegal to make copies of any software media provided to you by Invensys Systems, Inc. for any purpose other than those purposes mentioned in the software license.

Contents
Figures................................................................................................................................... vii

Tables..................................................................................................................................... ix

Preface.................................................................................................................................... xi

Purpose .................................................................................................................................... xi

Audience .................................................................................................................................. xi

Revision Information ............................................................................................................... xi

Reference Documents ............................................................................................................. xii

Terms and Definitions ........................................................................................................... xiii

1. Introduction ...................................................................................................................... 1

Overview of The MESH Control Network Architecture ........................................................... 1Switched Ethernet Characteristics ........................................................................................ 1The MESH Control Network Features ................................................................................ 2

Standard Configuration Features ..................................................................................... 2Security Enhanced Configuration Features ...................................................................... 2Loop Detection Policy (LDP) Deployed on The MESH Control Network ..................... 3Virtual Local Area Networks (VLANs) on The MESH Control Network ........................ 3

The MESH Control Network .............................................................................................. 4The MESH Control Network Topologies ....................................................................... 4

The MESH Control Network ................................................................................................. 12Network Example .............................................................................................................. 12The MESH Control Network Specifications ...................................................................... 15The MESH Control Network Workstations ...................................................................... 15

The MESH Control Network Ethernet Switches .................................................................... 16Advantages of Invensys-Supplied Switches .......................................................................... 16

Control Network Cabling ....................................................................................................... 17Category 5 Cabling ............................................................................................................ 18Fiber Optic Cabling ........................................................................................................... 18

Single Mode Cable ........................................................................................................ 19Multimode Cable .......................................................................................................... 19

The MESH Control Network Management Software Tool, NetSight® Console .................... 20Netsight Policy Manager .................................................................................................... 20Obtaining Network Management Software ........................................................................ 22

2. Site Planning ................................................................................................................... 23

Site Planning Overview ........................................................................................................... 23Network Considerations ..................................................................................................... 23

iii

B0700AZ – Rev L Contents

Traffic Considerations ........................................................................................................ 26Network Loading ............................................................................................................... 26Equipment Considerations ................................................................................................. 27Hardware Requirements ..................................................................................................... 28

Switch Utilization in the Standard Configurations ........................................................ 28Switch Utilization in the Security Enhanced Configurations ......................................... 28

Firmware Considerations .................................................................................................... 29

The MESH Control Network Design Rules ............................................................................ 29Standard Configuration Design Rules ................................................................................ 29

Non I/A Series Equipment ............................................................................................ 30Security Enhanced Configuration Design Rules ................................................................. 30Additional Guidelines for Planning Security Enhanced Configuration ............................... 34

Security Enhanced Configuration Design Considerations .............................................. 34

The MESH Control Network Design ..................................................................................... 36I/O Network Design Rules ................................................................................................. 36The MESH Control Network Topologies .......................................................................... 36

Standard Configurations ................................................................................................ 37Security Enhanced Configurations ................................................................................. 52I/O Network Topology Configurations ......................................................................... 58

Switch and Fiber Cable Budget and Loss ............................................................................ 59Fiber Cable Budget Cable and Loss ............................................................................... 59Fiber Optic Budgets ...................................................................................................... 59

The MESH Control Network Cabling ............................................................................... 59Switch and Converter Fiber Optic Cabling .................................................................... 59FCP270, ZCP270, FCM100Et and FCM100E Fiber Signal Cabling ............................ 63

Twisted-Pair Cabling ......................................................................................................... 66Null Hub ...................................................................................................................... 66

3. Installation (Cabling) ...................................................................................................... 67

Connecting The MESH Control Network Components ......................................................... 67Fiber Optic Cabling Guidelines .......................................................................................... 67Interconnecting Ethernet Switches ..................................................................................... 67

Uplink Port to Uplink Port ........................................................................................... 68RJ-45 Port to RJ-45 Port ............................................................................................... 70

Switch Configuration .............................................................................................................. 70

4. Maintenance.................................................................................................................... 71

The MESH Control Network Addresses ................................................................................. 71

General Troubleshooting Guidelines ....................................................................................... 72Characterize the Problem ................................................................................................... 72Determine Which Devices are Affected .............................................................................. 73Troubleshoot the Affected Devices ..................................................................................... 73

System Management Displays ................................................................................................. 74Accessing SMDH Switch Network Displays ....................................................................... 74Switched Network Display ................................................................................................. 76

Switch Equipment Change Display ............................................................................... 77

iv

Contents B0700AZ – Rev L

Switch Equipment Information Display ........................................................................ 78Switch Configuration Information Display .................................................................... 80

Switch Domain Display ..................................................................................................... 81Switch Ports Display .......................................................................................................... 82

Switch Port Equipment Change Display ....................................................................... 83Switch Port Equipment Information Display ................................................................ 84

Indicators ................................................................................................................................ 85Ethernet Switches ............................................................................................................... 85Media Converter ................................................................................................................ 85

Fiber Optic Cable Handling and Cleaning .............................................................................. 85Handling Fiber Optic Cable ............................................................................................... 85Cleaning Fiber Optic Cable ................................................................................................ 87

Contamination of Fiber Optic Connectors and Sockets ................................................. 87Contamination Prevention ............................................................................................ 88Contamination Removal ............................................................................................... 88Examples of Fiber Optic Connector Cleaning Products ................................................. 89

5. Combining Two or More MESH Control Networks ...................................................... 91

Overview ................................................................................................................................. 91Planning Stage .................................................................................................................... 91Station Addressing .............................................................................................................. 92Bridge Switch Functionality ............................................................................................... 92Topology Constants ........................................................................................................... 92

Combining Star Network Topologies ..................................................................................... 93

Combining Star with Tree Network Topology ........................................................................ 95

Combining Tree Network Topologies .................................................................................... 96

Combining Ring Network Topologies .................................................................................... 97

Appendix A. COMEX Fault Handling on The MESH Control Network............................ 99

COMEX Layers ...................................................................................................................... 99COMEX Applications Layer .............................................................................................. 99

Application Layer Protocol and Timers ......................................................................... 99Transport Layer ................................................................................................................ 100

Transport Layer Protocol and Timers .......................................................................... 102Network Layer ................................................................................................................. 103

Operation .................................................................................................................... 103Logical Link Control Layer .............................................................................................. 103

Transmit Operation .................................................................................................... 103Receive Operation ....................................................................................................... 103

MAC Layer ...................................................................................................................... 104

Appendix B. The MESH Network Fault Handling ........................................................... 105

LINK .................................................................................................................................... 105

“PORT TEST” Packets ......................................................................................................... 105

v

B0700AZ – Rev L Contents

“LLC_PING” Test ................................................................................................................ 106

ZCP-FCM Communications ................................................................................................ 107

DIAGNOSTIC Information ................................................................................................. 108

Index .................................................................................................................................. 111

vi

Figures
1-1. Linear Topology ............................................................................................................ 51-2. Ring Topology (Standard Configuration Only) ............................................................ 61-3. Star Topology ............................................................................................................... 81-4. Inverted Tree Topology .............................................................................................. 101-5. Modified Inverted Tree Topology ............................................................................... 111-6. The MESH Control Network (Star Topology Shown) ................................................ 131-7. The MESH Control Network with an I/O Network ................................................... 142-1. Large Network (Security Enhanced Configuration) ..................................................... 332-2. Small Network (Standard Configuration) ................................................................... 382-3. Medium Network (Ring) (Standard Configuration) .................................................... 392-4. Medium or Large Network (Star) (Standard Configuration) ....................................... 402-5. Medium or Large Network (Double Star) (Standard Configuration) .......................... 412-6. Large Network - All Blades (Standard Configuration) ................................................. 422-7. Large Network - Inverted Stepped Tier Tree - All Blades (Standard Configuration) .... 442-8. Large Network - Blades and Low-Cost Switches (Standard Configuration) ................. 452-9. The MESH Control Network Tiers ............................................................................ 472-10. Root Switches Connected ........................................................................................... 482-11. Second Tier Connections ............................................................................................ 492-12. Third Tier Connections .............................................................................................. 502-13. Fourth Tier Connections ............................................................................................ 502-14. Labeling the Switches .................................................................................................. 512-15. Small Network (Security Enhanced Configuration) .................................................... 522-16. Star Topology (Security Enhanced Configuration) ...................................................... 532-17. Double Star Topology (Security Enhanced Configuration) ......................................... 542-18. Inverted Tree Topology (Security Enhanced Configuration) ....................................... 562-19. Modified Inverted Tree Topology (Security Enhanced Configuration) ....................... 582-20. Switch to Switch and Switch to Patch Panel Connections ........................................... 632-21. FCP270, ZCP270, FCM100Et and FCM100E Signal Cabling .................................. 653-1. Switch-to-Switch Fiber via Uplink Port ....................................................................... 693-2. Port-to-Port Connection via Fiber Optic Ports ........................................................... 693-3. Connecting Switches via RJ-45 Ports .......................................................................... 704-1. Accessing SMDH Switched Network Displays ............................................................ 754-2. SMDH Switched Network Display ............................................................................. 764-3. Switch Equipment Change Display ............................................................................. 774-4. Switch Equipment Information Display ...................................................................... 784-5. Switch Configuration Information Display ................................................................. 804-6. Switch Domain Display .............................................................................................. 814-7. Switch Ports Display - Typical .................................................................................... 824-8. Switch Equipment Change Display ............................................................................. 834-9. Switch Port Equipment Information Display - Typical ............................................... 844-10. SC Connector, Typical ............................................................................................... 864-11. Multimode MT-RJ Connector .................................................................................... 874-12. Multimode Duplex LC Connector .............................................................................. 874-13. Lint in Fiber Optic LC Socket .................................................................................... 88

vii

B0700AZ – Rev L Figures

5-1. Combining Two Star Network Topologies Into One Tree Network Topology (Before) ....................................................................................................................... 93

5-2. Combining Two Star Network Topologies Into One Tree Network Topology (After) 935-3. Combining a Star Network Topology with a Tree Network Topology Into One Tree

Network Topology (Before) ........................................................................................ 955-4. Combining a Star Network Topology with a Tree Network Topology Into One Tree

Network Topology (After) .......................................................................................... 955-5. Combining Two Tree Network Topologies Into One Tree Network Topology

(Before) ....................................................................................................................... 965-6. Combining Two Tree Network Topologies Into One Tree Network Topology

(After) ......................................................................................................................... 965-7. Combining Two (4) Tier Tree Network Topologies Into One (4) Tier Tree Network

Topology (After) ......................................................................................................... 975-8. Combining Ring Network Topologies ........................................................................ 98

viii

Tables
1-1. The MESH Control Network Specifications ............................................................... 151-2. NetSight Console Policies ........................................................................................... 201-3. NetSight Client Policy ................................................................................................ 211-4. NetSight Policy Bundle ............................................................................................... 211-5. NetSight Advanced Bundle ......................................................................................... 211-6. NetSight Single User Policy ........................................................................................ 221-7. NetSight A-La-Carte Plug-Ins ..................................................................................... 221-8. NetSight Appliance Policy ........................................................................................... 222-1. Fiber Cable Power Losses ............................................................................................ 592-2. Multimode Fiber Cables with LC Connectors ............................................................. 602-3. Multimode Fiber Cables with MT-RJ to ST Connectors ............................................ 602-4. Multimode Fiber Cable with LC and SC Connectors ................................................. 612-5. Copper Cables with RJ-45 Connectors ....................................................................... 612-6. Single Mode Fiber Optic Cable - Maximum Transmission ......................................... 612-7. Single Mode Fiber Optic Jumper Cables ..................................................................... 622-8. Fiber Optic Cables ...................................................................................................... 642-9. CAT5 Cable - Maximum Transmission Distance ........................................................ 662-10. Prefabricated CAT5 STP Cables with RJ-45 Connectors ............................................ 662-11. Null Hub .................................................................................................................... 663-1. Methods of Connecting Ethernet Switches ................................................................. 684-1. IP Address Assignments ............................................................................................... 714-2. Switch Equipment Change Display Actions ................................................................ 774-3. Switch Equipment Information Display Fields ............................................................ 784-4. Switch Configuration Information Display Fields ....................................................... 804-5. Switch Equipment Change Actions ............................................................................. 834-6. Switch Port Equipment Information Display Fields .................................................... 84A-1. Out of Sequence DT and Ack Packets ...................................................................... 101

ix

B0700AZ – Rev L Tables

x

Preface
PurposeThis document provides overview guidelines and requirements for designing, installing, and maintaining The MESH control network. Topics include:

♦ Site Planning

♦ Installation

♦ Maintenance.

Additional documentation provides the information specific to the hardware for The MESH control network. These books are listed in “Reference Documents” below.

For detailed and specific information on the Ethernet equipment, refer to the documentation sup-plied by the switch vendor. These documents may also be available on the IPS Global Client Sup-port web site at http://support.ips.invensys.com. The MESH control network documents for I/A Series® systems are available on The MESH Network Configuration Tool CD-ROM (K0173ZU).

AudienceThis book is intended for use by process engineering, operations, installation, and maintenance personnel. They are expected to have a working knowledge of Ethernet LANs and I/A Series con-figurations.

Revision InformationFor this revision of the document (B0700AZ-L), the following changes were made:

Chapter 1 “Introduction”

♦ Significantly expanded “Network Example” on page 12.

♦ Updated references to Invensys in “Advantages of Invensys-Supplied Switches” on page 16.

♦ Updated Figure 1-6 “The MESH Control Network (Star Topology Shown)” on page 13.

♦ Updated “The MESH Control Network Management Software Tool, NetSight® Console” on page 20 to reflect the NetSight® products currently offered.

Chapter 2 “Site Planning”

♦ Minor edits to “Switch Utilization in the Standard Configurations” on page 28.

♦ Added “I/O Network Design Rules” on page 36.

♦ Added “I/O Network Topology Configurations” on page 58.

xi

B0700AZ – Rev L Preface

Reference DocumentsThe following documents provide additional or related information to the hardware used in The MESH control network:

♦ A-Series (P0973BH/P0973BJ/P0973BK) Switches, Hardware and Software Configura-tion Instructions (B0700CH)

♦ The MESH Control Network Hardware Instructions for C-Series Switches (P0973BL/HA) (B0700CJ)

♦ The MESH Control Network Hardware Instructions for N-Series Switches (P0973AR/P0973AS/P0972YE) (B0700CK)

♦ V-Series (P0972WP/P0972YC) Switches, Hardware and Software Configuration Instruc-tions (B0700CL)

♦ E7 Chassis and 16-port Fiber (P0972MK/P0972MJ) Switches, Hardware and Software Configuration Instructions (B0700CM)

♦ I-Series (P0973GB) Industrial Switch, Hardware and Software Configuration Instruc-tions (B0700CN)

♦ Media Converter Installation and Configuration Guide for Control Networks (B0700CP)

The following documents provide additional or related information to The MESH control network concepts:

♦ The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA)

♦ The MESH Control Network Architecture (PSS 21H-7C2 B3)

♦ The MESH Control Network Ethernet Equipment (PSS 21H-7C3 B4)

♦ I/A Series System Definition: A Step-by-Step Procedure (B0193WQ)

♦ I/A Series Configuration Component (IACC) User’s Guide (B0400BP).

xii

Preface B0700AZ – Rev L

Terms and Definitions10Base-T 10 Mb twisted-pair Ethernet

100Base-TX 100 Mb twisted-pair Fast Ethernet

100Base-FX 100 Mb fiber optic Fast Ethernet

1000Base-LX IEEE 802.3z specification for Gigabit Ethernet over two strands of 50/125 or 62.5/125 micron core MMF or 9/125 micron core SMF fiber cable using long wavelength optical transmission.

1000Base-SX IEEE 802.3z specification for Gigabit Ethernet over two strands of 50/125 or 62.5/125 micron core MMF fiber cable using short wavelength optical transmission.

1000Base-ZX IEEE 802.3z specification for Gigabit Ethernet over two strands of 9/125 micron core SMF fiber cable using 1550 nm wavelength optical transmis-sion.

1000Base-T IEEE 802.3ab specification for Gigabit Ethernet using CAT5 copper Ethernet cable.

ANSI American National Standards Institute

Auto-Negotiation Signalling method allowing each node to select its optimum operational mode (e.g., speed and duplex mode) based on the capabilities of the node to which it is connected.

Backbone Another term for bus - refers to the main link that connects network nodes. The term is often used to describe the main network connections composing the network.

Beacon The packet type and packet on the network upon which the port disabling is enacted.

BPP Beacon Priority Policy - A role/service that allows for the Beacon packet to have the highest priority when propagating though the network. This ensures the Beacon packet will be transmitted back to the PBQ in a flooded switch.

BootP Bootstrap Protocol

Bridge Priority Value

The range of priority values used to determine which device is selected as the Spanning Tree root. This value can range from 0- 65535 for bridge priority mode 802.1d (decrement by 1) or from 0-61440 for bridge prior-ity mode 802.1t (decrement by 4096).

CAT5 Category 5 Twisted Pair Cable - such as 10Base-T, 100Base-TX and 1000Base-T.

xiii


CBP (Circuit Breaker Policy) a role/service that disables a port when a Beacon packet is received from an edge switch or from the tier below.

Circuit Breaker A policy rule that will disable a port that receives an incoming packet of an outgoing Beacon packet “Loop”.

Circuit Breaker PBQ/SBQ (CBPBQ/CBSBQ)

This is used to refer to policy rule that will disable an uplink port that interfaces two root switches that receives an incoming packet of an outgo-ing Beacon packet “Loop”. This function is a subset of the Circuit Breaker.

CLI Command Line Interface

Core Switch Refers to the main body of switches that provide the network with its backbone connections. A core switch can also be considered an “edge switch” in reference to the root; however the outer most edge switches within the network are normally not considered to be core switches.

CRC Cyclic Redundancy Check

CSMA/CD Carrier Sense Multiple Access/Collision Detection

Data Loop or Loop Path

Refers to a condition where data traverses a redundant path with no termination point.

DCE Data Communications Equipment (modem)

DSR Data Set Ready

DTE Data Terminal Equipment

DTR Data Terminal Ready

Edge Switch Refers to an outer switch in a network topology that is linked to the pri-mary root or backup root bridge switch directly in one to two tier configurations, and indirectly in three to four tier configurations.

ESD Electrostatic Discharge

FCS Frame Check Sequence

Fast Ethernet (FE) Set of Ethernet standards that carry traffic at the nominal rate of 100 Mbit per second.

FTM Frame Transfer Matrix

Full Duplex Transmission method that allows two network devices to transmit and receive concurrently, effectively doubling the bandwidth of that link.

GBIC Gigabit Interface Converter

HTTP Hypertext Transfer Protocol

xiv

Preface B0700AZ – Rev L

ICMP Internet Control Message Protocol

IEEE Institute of Electrical and Electronics Engineers

IEEE 802.3 Defines carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specifications.

IEEE 802.3ab Defines a media access method and physical layer specifications for 1000Base-T Gigabit Ethernet.

IEEE 802.3u Defines a media access method and physical layer specifications for 100Base-TX Fast Ethernet over CAT5 cable.

IEEE 802.3x Defines Ethernet frame start/stop requests and timers used for paused flow control on full-duplex links.

IEEE 802.3z Defines a media access method and physical layer specifications for 1000Base Gigabit Ethernet.

IGMP Internet Group Management Protocol, used to establish host member-ships in particular multicast groups on a single network.

IOC Input/Output Controller (part of the Z-Module Control Processor (ZCP270))

IOM Input/Output Module

IP Internet Protocol

LAN Local Area Network

LDP Loop Detection Policy (Refer to The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA) for more information.)

LED Light Emitting Diode

MAC Media Access Control

MDI Media Dependent Interface or Media Device Interface

MIB Management Information Base

MMF Multi-mode Fiber cable

NEM Network Expansion Module

PBQ Primary Beacon Queryer - The switch with the lowest IP address and with the IGMP “Beacon” enabled.

Policy A group of rules which a network device uses to make forwarding, block-ing or port-disable decisions.

xv


RFC Request for Comment

RMON Remote Monitoring

Role A collection of services

RSTP Rapid Spanning-Tree Protocol (IEEE 802.1w standard)

Rule Hit An action when a packet classifier finds the packet.

Rules Packet classifiers that are used to identify packet types on the network.

RXD Receive Data

SBQ Secondary Beacon Query - The switch with the second lowest IP address and with the IGMP “Beacon” enabled.

Service A collection of Rules

SFP Small Form Factor Pluggable (Type of Mini-GBIC)

SMF Single-mode Fiber cable

SNMP Simple Network Management Protocol

STP Spanning-Tree Protocol (IEEE 802.1d standard)

TCP/IP Transmission Control Protocol/Internet Protocol

TDR Transient Data Recorder

TFTP Trivial File Transfer Protocol

TXD Transmit Data

UTP Unshielded Twisted Pair

VLAN Virtual Local Area Network

xvi

1. Introduction
This chapter provides an introduction to the concepts and equipment used in The MESH control network.

Overview of The MESH Control Network Architecture

The MESH control network is a switched Fast Ethernet network based on IEEE 802.3u (Fast Ethernet) and IEEE 802.3z (gigabit Ethernet) standards. The MESH control network consists of a number of Ethernet switches connected in a MESH configuration.

The MESH control network configuration allows high availability by providing redundant data paths and eliminating single points of failure caused by component link failures. The flexibility of the architecture allows you to design a network configuration that fits the needs of the control sys-tem. Configurations can be as simple as a workstation and controller connected with a single pair of switches, or as complex as a multi-switch, fully meshed control network, communicating at speeds up to 1 gigabit per second.

The MESH control network architecture integrates powerful control stations and workstations in a 100 Mb/1 Gb Ethernet network. These control stations, workstations and networks comprise scalable systems for process monitoring, process control and integration with industrial informa-tion management systems.

High speed, coupled with redundancy and peer-to-peer characteristics, provides high perfor-mance and superior security. Station interfaces to redundant Ethernet switches ensure secure communications between the stations. Station interfaces can use single paths but this compro-mises the security of The MESH control network.

NOTEAll graphics of switches and media converters in this document are intended as generic illustrations of networking concepts and do not necessarily reflect the cur-rently offered products.

Switched Ethernet CharacteristicsStandard Fast Ethernet switches and fiber optic/copper cabling provide versatile solutions for building MESH networks using industry standard protocols. The 16-port or larger managed Ethernet switches used in The MESH control network allow connection of multiple control sta-tions, workstations and other Ethernet switches. Unmanaged switches are not supported by The MESH control network because they offer no redundancy and you cannot run diagnostics if the switch should fail.

1

B0700AZ – Rev L 1. Introduction

The MESH Control Network FeaturesTwo types of network configuration methods are available in The MESH control network - Standard Configuration and Security Enhanced Configuration.

Standard Configuration FeaturesThe Standard Configuration of The MESH control network provides the following features.

♦ System scalability by interconnecting Ethernet switches with 16-ports or more in a linear, ring, star, inverted tree or modified inverted tree network topology (configura-tion). The topology is dependent on the network site requirements.

♦ Ethernet switches connected in a MESH configuration with up to 1920 I/A Series stations

♦ Support for Fast Ethernet (100 Mb) and Gigabit (uplink only) Ethernet (1000 Mb)

♦ Modular uplinks to high-speed backbones using 1 Gb 1000Base-T, 1000Base-SX, 1000Base-LX and 1000Base-ZX standards

♦ Full-duplex operation based on the IEEE 802.3 standards

♦ Rapid Spanning Tree Protocol (RSTP - IEEE 802.1w) which manages redundant paths, prevents loops, and provides high speed convergence time for a network

♦ Network management and configuration via local port or Web access for various switches

♦ System Management software for monitoring the health of the control system and managing equipment in the system

♦ Software in every station that manages redundant Ethernet ports in response to net-work faults

♦ High speed response to network and station faults to provide a highly reliable redun-dant network.

Security Enhanced Configuration FeaturesThe Security Enhanced Configuration is now available to provide additional features. Due to the recent advances in switch network technologies, the Security Enhanced Configuration of The MESH control network offers improvements in network security, Loop Detection and many additional features not offered in the Standard Configuration.

The Security Enhanced Configuration deploys specific network topologies and switch configura-tions that allow for advanced network loop detection in the event of a RSTP failure. This advanced network loop detection minimizes the potential for a single point of failure that will degrade communications between devices in the network.

Deploying the advanced network loop detection is accomplished by carefully designing the net-work and correctly deploying the Loop Detection Policy (LDP) algorithms required for the net-work design, and by following the network configuration requirements provided in this document.

2

saravana.prabu

Comment on Text

Sequrity enhanced network deviate from standard netwrok by menas on Loop Detecion policy

1. Introduction B0700AZ – Rev L

NOTEWhen designing a Security Enhanced Configuration network, each device/switch within the network is required to be connected to two different switches within the network. If the network is constructed with less than two connections between devices/switches, data traffic could be disrupted if any single device fails or there is a fault that causes the loop detection algorithm Rule Hit.

Loop Detection Policy (LDP) Deployed on The MESH Control NetworkDue to the design of the Standard or Security Enhanced Configurations of The MESH control network, redundant links form physical loops in the network and are controlled (Blocked) by Rapid Spanning Tree Protocol (RSTP), creating a logical loop-free network. In a Security Enhanced Configuration in addition to RSTP, Loop Detection (LDP) is deployed to block redundant loops that could occur in the event of an RSTP or Data Loop (storm) failure.

A loop is determined by establishing a well-known data path and its source. To establish a known path, the concept known as the “Beacon” is developed. The Beacon routinely sends out an IGMP data packet. When the data packet is seen at an unexpected source port, the assumption is that a loop occurred and an action (Rule) needs to occur. A switch’s port deployed with “Circuit Breaker” will disable the first port on which the incorrectly sourced packet is received.

Recent chassis switches, such as the N-Series Platinum (DFE) switches (P0973BQ, P0973BR, P0973BT, and P0973BS), offer advanced packet switching services that can scope data packets beyond the source and destination MAC-address. By looking at other data points in the packets, the switch can make decisions on which of these data points to mark a packet on. Once a particu-lar packet is identified, the switch can take action on it. The action of interest is disabling a looped port. Disabling this looping port maintains a loop-free network. The switches alert the network administrator with SNMP traps and syslog messages. These should be acted upon to 'fix' the net-work loop in a timely manner. When disabled by the LDP, a disabled port can be monitored by SMDH via a link down trap. Other methods of monitoring and management of ports can be accomplished by utilizing the switch's CLI port or NetSight Policy Manager.

Virtual Local Area Networks (VLANs) on The MESH Control NetworkA VLAN acts like an ordinary Local Area Network (LAN), but in a VLAN, connected devices do not have to be physically connected to the same segment. The VLAN allows devices located in separate areas or connected to separate ports to belong to a single VLAN group. Devices that are assigned to such a group will send and receive broadcast and multicast traffic as though they were all connected to a common network. VLAN-aware switches isolate broadcast, multicast, and unknown traffic received from VLAN groups, so that traffic from stations in a VLAN are con-fined to that VLAN.

Additional details about VLANs are provided in The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA revision F or later), in Chapter 10 “VLANs Usage on The MESH Control Networks” and Appendix D “Understanding Virtual Local Area Networks (VLANs)”.

3

saravana.prabu

Highlight

saravana.prabu

Highlight

saravana.prabu

Highlight


The MESH Control NetworkThe MESH control network is designed to provide multiple communication paths between any two devices or stations connected to the network. This network architecture provides very high availability, while reducing network complexity, cost, and maintenance requirements.

The MESH Control Network TopologiesThere are several basic MESH control network topologies supported by the I/A Series system in each of the Standard and Security Enhanced Configurations. These are:

♦ Standard Configuration

♦ Linear

♦ Ring

♦ Star

♦ Inverted tree

♦ Modified inverted tree

♦ Security Enhanced Configuration

♦ Linear

♦ Star

♦ Inverted tree


NOTEWhen deploying the Security Enhanced Configuration, The MESH control net-work should be constructed using one of the four enhanced topologies listed above. The ring topology should never be used when deploying this configuration.

Each configuration/topology listed has unique features and the one chosen for a particular net-work depends on the specific requirements of the site or installation. The following diagrams pro-vide examples of the different topologies as well as recommendations on where they might be used.

4


Linear Configuration

The linear configuration is appropriate for a small, two switch network, as shown in Figure 1-1. In this example a failure of any one component in The MESH control network does not affect the operation of the remaining components. The linear configuration does not require a root or back-up switch configuration. Large chassis switches with hundreds of ports can be used in a linear configuration to create a large control system. A multiple switch linear configuration (more than two) is not supported due to its lack of network redundancy, which can result in loss of communi-cation between two devices within the network.

Figure 1-1. Linear Topology

5


Ring Topology

This topology is suited to networks containing three to seven switches. As shown in Figure 1-2, each switch has connections to the two adjacent switches. In the event of a failed switch, the ring is broken and the network assumes the characteristics of the linear topology shown in Figure 1-1.

There is a limit of seven (7) switches between any two devices on the network. This limit is imposed by the I/A Series system and cannot be exceeded.

Figure 1-2 illustrates a network composed of six managed switches configured in a ring. A net-work in this configuration is able to handle a single component failure and still maintain its integ-rity. The ring configuration does not require a root or backup root switch configuration.

The ring topology cannot be configured with the Security Enhanced Configuration. When LDP is deployed in a ring topology, the loop detection algorithm will run before RSTP can block the port resulting in an uplink port being disabled. The end result will produce a multi-ple switch linear topology which is not a supported configuration. When designing Security Enhanced Configurations, use one of the four supported topologies (on page 4).

Figure 1-2. Ring Topology (Standard Configuration Only)

6


Star Topology

The star topology is the preferred topology for all networks, provided that geographical con-straints allow this topology. In a star topology, the switches at the outside edge of the network have connections to each of the two root switches. The root switches are connected to each other and the edge switches. Redundant data paths allow the network to continue to operate if any one component fails.

Figure 1-3 illustrates a star network containing ten edge switches and two root switches. In a Stan-dard Configuration star topology utilizing Gold Series blades, as many as 40 edge switches can be connected to the Chassis switch using 1 Gb uplinks. When utilizing Platinum Series blades, as many as 166 edge switches can be utilized.

For a Security Enhanced Configuration star topology (Policy enabled switches only) a maximum of 166 edge switches can be connected to the Chassis switch using 1 Gb uplinks. An inverted tree topology can be considered if there is a larger number of edge switches required (up to 250 switches).

Refer to the table “Qualified Switch Standard/ Security Enhanced Configuration Compatibilities Matrix” in The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA) for a list of switches capabilities.

7

saravana.prabu

Note

Need to check


Figure 1-3. Star Topology

8


Inverted Tree Topology

The inverted tree topology is suited to very large networks with specific physical constraints. In this topology the switches are arranged in tiers, with the root switches in the top tier and up to three tiers below them. The root switches (Tier 1) are the only switches in the network that have connections between switches on the same tier; all other switches have two connections to switches in the tiers above them. This topology is supported in both the Standard and Security Enhanced Configurations.

When deployed in a Security Enhanced Configuration, all switches within the network must be switch types that support Loop Detection (LDP). Refer to the table “Qualified Switch Stan-dard/Security Enhanced Configuration Compatibilities Matrix” in The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA) for a list of the switches applicable to either of these configurations.

An inverted tree network topology is illustrated in Figure 1-4.There can be no more than four-tiers of switches (including the root) in order to comply with the I/A Series system requirement limiting the number of switches between devices to seven.

9


Figure 1-4. Inverted Tree Topology

Modified Inverted Tree Topology

The modified inverted tree topology is suited for very large enhanced networks with specific phys-ical constraints and requirements. The modified inverted tree topology allows for standard quali-fied (lower cost) switches to be utilized at the outer edge, which allows for larger networks to be deployed at a lower cost. In this topology, the switches are arranged in tiers, with the root switches in the top tier with up to three tiers below them. The root switches (Tier 1) are the only switches in the network that have connections between switches on the same tier; all other switches have two connections to switches in the tiers above them. All outer edge switches are interfaced to the network on different tiers (e.g. tier 3 and tier 4). By doing this, all end devices with redundant

Note: Primary and Backup Root Switches have twoconnections.

10


connections interface to the network on different tiers. A modified inverted tree network topol-ogy is illustrated in Figure 1-5.

There can be no more than four-tiers of switches (including the root) in order to comply with the I/A Series system requirement limiting the number of switches between devices to seven.

Figure 1-5. Modified Inverted Tree Topology

11


The MESH Control Network

Network ExampleThe MESH control network utilizes qualified Fast Ethernet switches which are configured to form a highly robust redundant network. Figure 1-6 on page 13 shows an example with several I/A Series stations and Control Processors connected redundantly to The MESH control network.

Workstations are also redundantly connected to the Fast Ethernet switches.

The Z-module Control Processor 270 (ZCP270) main processor’s Ethernet ports connect to The MESH control network, where the ZCP270 I/O controller (IOC) ports have the option of con-necting directly to The MESH control network or a dedicated I/O network.

The Fieldbus Communications Module (FCM100Et or FCM100E) provides the interface between the ZCP270 and the FBMs and connects to The MESH control network or a dedicated I/O network.

When designing The MESH control network architecture, the following concerns should be addressed:

♦ For critical I/O communications, which in this context can be defined as I/O applica-tions that cannot allow for a disruption of fieldbus communications over a five second span, it is recommended that the ZCP270 IOC and FCM I/O be placed on a separate network.

♦ For non-critical I/O communications, the FCM100s and ZCP IOC ports can con-nect directly to The MESH. Whenever possible, it is recommended that the ZCP IOC ports and FCMs be attached to the same switches within the network, keeping the ZCP IOC to FCM communications local to the switch. (See Figure 1-6.)

If a dedicated I/O network is to be employed, a simple linear topology is used and switch selection should be per the standard guidelines for The MESH, where the ZCP270 I/O controller (IOC) ports A/B and FCM I/O ports A/B (Fieldbus A and Fieldbus B) A side and B side are separated on independent networks. It is also possible to have multiple I/O networks, such as having an I/O network for each ZCP IOC and its FCMs.

WARNING!Switches on a dedicated I/O network cannot be seen in SMDH or System Manager. However, Fieldbus A and Fieldbus B errors are indicated in a normal manner via the “Sys” Key and are visible in SMDH or System Manager.Also, a dedicated I/O network cannot be used for FBMs using the GPS SOE/TDR time sync package (see Time Synchronization User's Guide (B0700AQ)).

For more information on configuring a dedicated I/O network, refer to “I/O Network Design Rules” on page 36 and “I/O Network Topology Configurations” on page 58.

12


Figure 1-6. The MESH Control Network (Star Topology Shown)

The MESH

ETHERNETSWITCHES

ZCP270

FCP270

TO/FROMPROCESS

CONTROL NETWORK

TO/FROMPROCESS

DIN RAIL

FCM100Et

WORKSTATIONS

INFORMATION NETWORK

BASEPLATE

SPLITTER/COMBINER

FBMFBM

(NOT USED WITH FCM100E)

SPLITTER/COMBINERS

DIN RAILBASEPLATE

or FCM100E

13


Figure 1-7. The MESH Control Network with an I/O Network

ZCP270

SPLITTER/COMBINERS

TO/FROM

PROCESS

DIN RAIL

FCM100Et

BASEPLATE

SPLITTER/COMBINER

FBM

(NOT USED WITH FCM100E)

or FCM100E

The MESHCONTROL NETWORK

I/O NETWORK

A

B

14


The MESH Control Network Specifications

For more information, refer to the following Product Specifications Sheets:

♦ The MESH Control Network Architecture (PSS 21H-7C2 B3)

♦ The MESH Control Network Ethernet Equipment (PSS 21H-7C3 B4).

The MESH Control Network WorkstationsI/A Series workstations can be connected to The MESH control network. These workstations provide host services to fault-tolerant control processors. In addition, the workstation provides the operator interface for the display of graphic and textual information.

Each workstation connects to the switches in The MESH control network by way of copper or fiber interface cards in the workstation. Two Network Interface Cards (NIC) are offered:

♦ 100Base-FX

♦ 10/100Base-TX

The workstation can be directly connected to Ethernet switches or through the use of a media converter, described in Media Converter Installation and Configuration Guide for Control Networks (B0700CP). In addition, the built-in Ethernet interface on the workstation’s motherboard can be used to interface to a plant information network.

Table 1-1. The MESH Control Network Specifications

Specifications

Number of I/A Series stations connected to The MESH control network

1,920 I/A Series stations including switches, maximum (FCM100Ets and FCM100Es are not included in count). Up to 250 switches.

Number of Ethernet switches between any two stations

Seven maximum

Number of IP addresses 10,000 maximum includes switches, controllers, workstations and FCM100Ets and FCM100Es.

VLANs on The MESH control network Six (6) configurable VLANs are supported on The MESH Control Network

Standards Supported 100 Mb Full-duplex operation for fiber optic and copper cable.Modular uplinks using 1 Gb 1000Base-T, 1000Base-SX and 1000Base-LX standards

Speeds Supported Fast Ethernet (100 Mb) and uplink Gigabit Ethernet (1000 Mb)

Protocols Used Rapid Spanning Tree Protocol (RSTP - IEEE 802.1w), 802.3, 802.3ad

Cable Lengths - Interconnecting stations or Ethernet switches

CAT5:100Base-TX or 1000Base-T; 100 m (328 ft) maximumFiber optic:100Base-FX; 2 km (6,560 ft) maximum (MMF)1000Base-SX; 275 m (900 ft) maximum (MMF)1000Base-LX/LH; 2 km (6,560 ft) maximum (MMF)1000Base-LX; 10 km (6.2 mi) maximum (SMF)1000Base-ZX; SMF, 80 km (49.7 miles) maximum

Cable Lengths - Total connection length allowed between switches

Single mode fiber (SMF), 80 km (49.7 mi) maximumMultimode fiber (MMF), 2 km (6,560 ft) maximum

MESH Control Network Distance It is possible to extend The MESH control network distance using various third-party network equipment and extenders. It should be noted that the total network delays between two end devices should not exceed 100ms roundtrip.

15


The MESH Control Network Ethernet SwitchesA switch is an active multiport network and bridge device that provides a separate collision domain for each port, and uses Media Access Control (MAC) layer to direct network packets to the appropriate station or switch. This allows multiple simultaneous communications among net-work devices connected to the switch.

The MESH control network utilizes standard commercial off-the-shelf Ethernet switches to allow you to configure your system to meet your functional, performance and plant requirements. Switches listed in “Reference Documents” on page xii have been tested and qualified by Invensys for use with I/A Series products. Other Fast Ethernet switches from other vendors may be allowed to be on The MESH control network. Using non-qualified switches may cause unpredictable fail-ures or responses. The list of vendors and their qualified switches can be obtained from the Global Client Support Center (Global CSC) web site at http://support.ips.invensys.com.

Refer to the documentation included with your Invensys qualified Ethernet switches for details of their capabilities.

NOTEThe MESH control network was designed and tested for operation with the Ether-net switches listed in “Reference Documents” on page xii. The network may operate with similar, off-the-shelf equipment, but Invensys Systems, Inc. is not responsible for any system malfunctions that may occur if such equipment is used. If you use your own network:1. You must meet the bandwidth requirements for the I/A Series equipment you have chosen (1 Gb for uplinks and 100 Mb for ports).2. There can be no Layer 3 inter-network devices (for example, routers) between any I/A Series equipment.3. Typically a failover time of less than 1 second is achievable using Fast Ethernet switches qualified and supplied by Invensys and configured in accordance with I/A Series documentation.

The network management module for the Fast Ethernet switches, provides a menu-driven or Command Line Interface (CLI) system configuration program with management capability. See the documentation included with your Invensys qualified Ethernet switches and the vendor man-uals listed in “Reference Documents” on page xii for further details.

Advantages of Invensys-Supplied SwitchesThe MESH control network requires switches purchased from Invensys. This provides a number of advantages, described below.

♦ Can customers include Ethernet switches from other suppliers for use in The MESH control network?

Only Invensys-supplied Ethernet switches have been qualified for use in the I/A Series system. Other switches may or may not work and may not meet the performance specifications required for a secure, reliable high performance in The MESH control network.

16


♦ Why won't other switches provide the same performance?

The Invensys-supplied switch is off the shelf equipment and has been optimized to meet the stringent performance specifications for The MESH network. For example, the Rapid Spanning Tree Protocol algorithm that manages traffic paths is optimized to meet the requirement for network recovery on failure of a root switch in less than one second. This level of performance is much better than is typically achievable with other vendor's hardware and RSTP implementation.

♦ Can customers purchase switches from third parties with the correct certified firm-ware to work with the I/A Series system?

Although switches qualified for use with I/A Series systems can also be purchased from third parties, they will not necessarily have been furnished with the correct firmware version that has been qualified. The I/A Series Switch Configurator Application Software (SCAS), supplied with the Invensys-supplied switch, is also designed to configure switches qualified by Invensys, making configuration easier, quicker, more reliable and facilitates troubleshooting and configuration verifica-tion.

♦ How could a user get the correct firmware?

They would need to contact Invensys and either purchase replacement firmware for each switch on a one time charge basis or purchase an Invensys support con-tract which would cover the cost of replacement firmware.

♦ Is the firmware provided by Invensys for The MESH network switches exclusive to Invensys-supplied switches?

Changes to the firmware used in I/A Series systems are included in the standard product.

Periodically, changes are made to the firmware; future versions of the firmware which have not been qualified by Invensys may or may not be compatible with I/A Series systems.

Control Network CablingThree different types of cabling of various lengths may be used:

♦ Shielded twisted-pair 100Base-T CAT5 cabling - 100 m (328 ft.) maximum

♦ Shielded twisted-pair 1000Base-T CAT5 cabling - 100 m (328 ft.) maximum

♦ Multimode fiber optic cabling - 2 km (1.25 mi) maximum for 100Base-FX

♦ Multimode fiber optic cabling - 275 m (900 ft) maximum for 1000Base-SX

♦ Multimode fiber optic cabling - 550 m (1800 ft) maximum for 1000Base-LX

♦ Multimode fiber optic cabling - 2 km (1.25 mi) maximum for 1000Base-LX/LH

♦ Single mode fiber optic cabling - 10 km (6.21 mi) maximum for 1000Base-LX.

♦ Single mode fiber optic cabling - 80 km (49.68 mi) maximum for1000Base-ZX/ELX.

17


Category 5 CablingThere are two basic configurations of Category 5 (CAT5) cables:

♦ Straight-through cables: cables used to connect workstations to Ethernet switches, and media converters to Ethernet switches.

♦ Crossover cables (also called a null hub): cables used to interconnect Ethernet switches.

Some switches have auto MDI/MDI-X ports (for example, 24-Port Copper) which provides the crossover and do not require a crossover cable.

NOTEAll category 5 Cabling must be of shielded type for optimal interference mitigation.

Normally, when one switch is connected to another, the transmit and receive wires must be crossed over, such that the transmit wires from switch #1 connects to the receive wires from switch #2, and vice versa. Crossover cables are used much less frequently than straight-through cables. A straight through cable can be used as a crossover cable if used in conjunction with a null hub cable adapter (P0971PK).

A null hub is a very short cable that has a male RJ-45 connector on one end and a female RJ-45 connector on the other. The transmit and receive wires are reversed, so when it is connected to the end of a straight-through cable, the resulting cable system can act as a crossover cable. The null hub is used to interconnect switches using straight-through CAT5 when neither switch is equipped with auto MDI/MDI-X ports (crossover port).

Fiber Optic CablingFiber optic cable is used to connect workstations to Ethernet switches and to make connections between Ethernet switches. The fiber optic cable’s electrical isolation characteristics provide pro-tection from voltage differentials and ground loops and permit communication installations to pass through areas where intrinsically safe operation is required. The fiber optic cable is unaf-fected by electrical noise such as EMI and RFI and can be installed even in the following cases:

♦ Areas containing rotating machinery, arc welders, and so forth

♦ Cable trays containing high voltage power lines

♦ Outdoor areas exposed to lightning hazards (with appropriately rated cable)

♦ Areas containing strong magnetic fields

♦ Longer distances than twisted pair cable.

Two different types of fiber optic cable may be used in The MESH control network:

♦ Single mode cable

♦ Multimode cable (switch-to-device connections and uplink connections).

18

saravana.prabu

Highlight


NOTESingle mode and multimode fiber optical devices are not compatible. Both devices being connected (and the cable) must be of the same type in order to ensure proper operation. In certain limited cases (connecting Ethernet switch uplink ports, for example), a mode conditioning cable may be employed so that multimode fiber cable can be used with a single mode device. Refer to the diagrams in the “Ethernet Switch Interconnection Diagrams” in the documentation included with your Invensys qualified Ethernet switches for specific information.

NOTEThe fiber optic cables mentioned in this document may require additional mechanical protection, particularly when run between different enclosures.

Single Mode CableIn the control network, single mode fiber optic cable can be used to connect Ethernet switches to each other through each switch’s uplink port. The uplink ports of each of the switches being con-nected must be designed for single mode operation. Refer to the “Ethernet Switch Interconnec-tion Diagrams” in the documentation included with your Invensys qualified Ethernet switches for information on uplink modules for use with single mode fiber optic cable.

Multimode CableMultimode fiber optic cable is employed in different situations in The MESH control network:

♦ Controller to Switch - A controller connects through splitter/combiner(s) to the Ethernet switch(es) 100Base-FX port(s) using a multimode fiber optic cable termi-nated with MT-RJ connectors. The connection is LC on one end and MT-RJ on the other end between the switch and the splitter. Ethernet switch(es) without fiber optic port(s), can use media converter(s) between the switch(es) and the splitter/com-biner(s) to enable a connection.

♦ Field Communications Module (FCM) to Switch - A redundant FCM connects through splitter/combiners to the Ethernet switches 100Base-FX ports using a multi-mode fiber optic cable terminated with MT-RJ connectors. The connection is LC on one end and MT-RJ on the other end between the switch and the splitter. Ethernet switch(es) without fiber optic port(s), can use media converter(s) between the switch(es) and the splitter/combiner(s) to enable a connection. A single FCM module does not use a splitter/combiner and connects directly to a switch via an LC/MT-RJ fiber cable.

♦ Workstation to Ethernet Switch - A workstation is capable of connecting directly to Ethernet switch’s 100Base-FX port using a multimode fiber optic cable terminated with MT-RJ connectors. An Ethernet switch without a fiber optic port, can use two media converters between the switches and the workstations. Connection is based on NIC type and switch type. Media converters can be used when NICs do not match switch ports.

19


♦ Ethernet Switch to Ethernet Switch - Ethernet switches that are equipped with multi-mode fiber uplink ports may be connected directly to each other. The fiber optic cable should be terminated with MT-RJ type or LC type connectors, depending on the switch interface modules.

The MESH Control Network Management Software Tool, NetSight® Console

The NetSight® Console enables enterprise-wide management of The MESH control network to provide network-wide monitoring and troubleshooting, such as device discovery, topology map-ping, and event management.

The enterprise-level command and control console provides:

♦ Multi-element management approach to facilitate the abstraction of complex network policy into everyday business language

♦ System-level monitoring, and troubleshooting capabilities such as device discovery, event management, and logging.

NetSight Plug-ins use the advanced features while reducing administrative burden and lowering total cost of ownership. The Plug-in applications include the NetSight Policy Manager.

Netsight Policy ManagerAn advanced plug-in application for NetSight®, the NetSight Policy Manager enables simple management of complex network security policies to greatly enhance reliable network connectiv-ity. It provides the following.

♦ Role-based enterprise management

♦ Defines Roles, Rules and Services of the network

♦ Matches the role of the device with available network services

♦ Facilitates a distributed firewall to all edge points in the network

♦ Automated capabilities - offers ease of implementation, administration and troubleshooting

♦ Complex policy management - provides an Audit Trail (event log)

The following tables list the various NetSight Policy Manager software available.

Table 1-2. NetSight Console Policies

Part Number Detailed Description

NS-CON-50 NetSight Console 50 Devices (50 device license for 1 server plus 3 concurrent user licenses)

NS-CON-U NetSight Console Unrestricted (Unrestricted device license for 1 server plus 25 concurrent user licenses)

NS-CON-U-UG NetSight Console 50 to Unrestricted Upgrade

20


Table 1-3. NetSight Client Policy


NS-USER NetSight User (Add 1 concurrent user license to existing NetSight server)

Table 1-4. NetSight Policy Bundle


NS-PB-50 NetSight Policy Bundle 50-devices (50 device Console license for 1 server plus 3 concurrent users, Policy Manager and Policy Control Console)

NS-PB-U NetSight Policy Bundle Unrestricted (Unrestricted device Console license for 1 server plus 25 concurrent users, Policy Manager and Pol-icy Control Console)

NS-PB-U-UG NetSight Policy Bundle 50 to Unrestricted Upgrade

Table 1-5. NetSight Advanced Bundle


NS-AB-50 NetSight Advanced Bundle 50-devices (50 device Console license for 1 server plus 3 concurrent users with Policy Manager, Policy Control Console, Automated Security Manager, Inventory Manager, and NAC Manager)

NS-AB-U NetSight Advanced Bundle Unrestricted (Unrestricted device Con-sole license for 1 server plus 25 concurrent users with Policy Manager, Policy Control Console, Automated Security Manager, Inventory Manager, and NAC Manager)

NS-AB-U-UG NetSight Advanced Bundle 50 to Unrestricted Upgrade

NS-AB-50FT NetSight Advanced Bundle 50-devices FT (50 device Console license for 1 server plus 3 concurrent users, Policy Manager, Policy Control Console, Automated Security Manager, Inventory Manager, NAC Manager, a redundant NetSight license for fault tolerance (manual failover), Includes Lab License)

NS-AB-UFT NetSight Advanced Bundle Unrestricted FT (Unrestricted device Console license for 1 server plus 25 concurrent users, Policy Manager, Policy Control Console, Automated Security Manager, Inventory Manager, and NAC Manager, a redundant NetSight license for fault tolerance (manual failover), includes Lab License)

NS-AB-UFT-UG NetSight Advanced Bundle 50 to Unrestricted FT Upgrade

21


Obtaining Network Management SoftwareTo obtain Invensys pricing discounts, use the following contact information:

A free evaluation copy of the software can be downloaded at:

http://www.enterasys.com/products/management/NSA-LIC/

Table 1-6. NetSight Single User Policy


NS-SU-10 NetSight Single User 10-devices (10-device, single client (on same machine as server) license for Console, Policy Manager, and Inven-tory Manager)

Table 1-7. NetSight A-La-Carte Plug-Ins


NS-ASM NetSight Automated Security Manager (NetSight Automated Secu-rity Manager. Requires existing NS-CON-50 or NS-CON-U license)

NS-IM NetSight Inventory Manager (NetSight Inventory Manager. Requires existing NS-CON-50 or NS-CON-U license)

NS-NAC NetSight NAC Manager (NetSight NAC Manager. Requires existing NS-CON-50 or NS-CON-U license)

NS-PM NetSight Policy Manager (NetSight Policy Manager. Requires exist-ing NS-CON-50 or NS-CON-U license)

NS-LAB NetSight Lab License (Non-production use, one-time fee inclusive of maintenance. 50 device license for 1 server plus 2 concurrent users with Policy Manager, Automated Security Manager, Inventory Man-ager, NAC Manager, Policy Control Console. Requires existing NS-CON-50 or NS-CON-U license)

Table 1-8. NetSight Appliance Policy


SNS-NSS-A NetSight Appliance (Hardware only, requires separate NetSight license)

Holly O'Gara978-878-4579 (U.S. Number)[email protected]

OR Christine Leblanc978-684-1559 (U.S. Number)[email protected]

Enterasys NetworksCorporate Headquarters50 Minuteman RoadAndover, MA 01810U.S.A

22

2. Site Planning
This chapter describes the steps that should be taken and the options that should be considered when planning The MESH control network.

The design of each instance of The MESH control network is different, and depends on the needs and requirements of the individual site. The following paragraphs provide the information neces-sary to help network designers to plan The MESH control network that meets the needs of their specific site.

Site Planning OverviewThe MESH control network allows communication between the control stations, workstations, and Field Communications Modules (FCMs) connected to the network. This network is formed by interconnecting the control stations and workstations through the use of fiber optic or copper cable, Ethernet switches and, if necessary, media converters. Although copper cable can be used, fiber cable is recommended for industrial networks. In a properly constructed network, all the sta-tions on the network are able to communicate with each other. For a high degree of reliability, The MESH control network should be constructed so that there are redundant signal paths between each device on The MESH control network.

Although the design of The MESH control network is driven largely by the site’s physical envi-ronment, traffic and organizational requirements, there are rules that must be followed when con-necting devices to form The MESH control network. The following two sections explain the factors affecting the physical and the traffic considerations of The MESH control network.

Network ConsiderationsThe physical location of equipment at the site influences The MESH control network design. To choose equipment and cabling appropriate to the requirements for the specific site, use the answers to the following questions, along with The MESH control network design rules and device/cable specifications in this chapter and Appendix A “COMEX Fault Handling on The MESH Control Network”.

♦ What is the maximum end-to-end distance between devices in The MESH control network?

The distance between the ends of The MESH control network may determine what kind of Ethernet switches are used and what kind of cabling is used between those switches. If individual cable runs are greater than a hundred meters, it is necessary to use fiber optic cable, due to its capability for transmitting signals over longer distances. All hardware must be considered when the network requires multiple hops; no low end standalone switches should be used in the network core.

It should be noted that the total network delays between two end devices should not exceed 100 ms.

♦ Will The MESH Control Network have VLANs?

23

B0700AZ – Rev L 2. Site Planning

NOTEIf the answer to this question is No, it is recommended that when setting up a first time installation of The MESH control network that VLAN 2 “I/A Control Ports” is deployed on all ports used for communications between I/A Series devices. (See the warnings below.)If a system is up and functional without VLAN 2 deployed across the network, a system shutdown is required to deploy VLANs. If VLAN 2 is deployed prior to system startup, additional VLANs can be added at a later date without system interference (shutdown).

♦ If yes, how many VLANs will The MESH control network require?

The MESH control network will support up to six VLANs, one (1) of which must be reserved for I/A Series system devices (VLAN 2 “I/A Control Ports”), and all I/A Series control devices shall be attached to VLAN 2 FE ports.

Only one I/A Series control system can be supported on The MESH control network.

There can be no duplicate MAC addresses across The MESH control network VLANs.

If VLANs are deployed, all switches in the network must have VLAN 2 enabled on the switch’s uplink ports.

WARNING!When VLANs are added to an existing installation of The MESH control network, if the I/A Series devices are on VLAN 1, they must be moved to VLAN 2, at which time communications between the I/A Series devices will be broken.

WARNING!All I/A Series devices must be connected to Device ports which have been assigned to VLAN 2 “I/A Control Ports”. If this is not done, the I/A Series devices on the network will not communicate correctly with each other. Refer to the “Flowchart for Assignment of Uplinks and Ports to VLAN” figure in the “Switch Configura-tion Parameters Dialog Box” section in The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA).

If VLANs are enabled, all switches in the network must have VLAN 2 “I/A Control Ports” set to ENABLED.

WARNING!If VLANs are to be utilized in The MESH control network, all switches within the network are required to be configured for all utilized VLANs. If a VLAN is config-ured on an outer edge switch and a core switch has not been configured for that VLAN (in the case where no port assignment is required), data from the outer edge switch VLAN will not propagate through the core switch.

24

2. Site Planning B0700AZ – Rev L

♦ What will the protocols be on the non-I/A Series VLANs?

VLANs other than VLAN 1 and 2 shall not utilize protocols other than TCP/IP and/or UDP/IP.

♦ What will be the loading on the FE device ports?

No VLAN Port shall exceed 50% loading.

♦ What is the non-I/A Series VLAN end device speed usage?

No end devices with transfer (Tx) and receive (Rx) speeds greater than 100Mb are supported.

♦ Will the switch be installed in an adverse environment?

For the requirements for industrial switches in The MESH control network, refer to I-Series (P0973GB) Industrial Switch, Hardware and Software Configuration Instructions (B0700CN) for the I-series Industrial switch specifications.

♦ How many control stations and workstations will be connected to The MESH con-trol network?

The number of control stations and workstations influences how many Ethernet switches and separate cables are included in The MESH control network. It may be wise to plan for expansion and growth when arriving at this number.

♦ How are the control stations and workstations distributed?

The physical distribution of control stations and workstations can have a bearing on the type and number of Ethernet switches, as well as the type of cabling used. If the control stations and workstations are clustered together, two larger capacity switches may be able to accommodate them all. If they are dispersed, several smaller switches may be a better solution. For large distances and noise protection, you should use fiber optic cable between switches, or between switches and con-trol stations, workstations, or FCMs. Cable routing between devices must also be considered.

♦ Will The MESH control network have redundant signal paths?

Yes, a control network with redundant paths is recommended for all control sys-tems.

♦ Through what kind of physical environment will the cabling pass?

Fiber optic cable is immune to magnetic fields and electrical noise, so it can be used in places where copper cable would be unreliable. If cabling must be routed through areas subject to high EMI or RFI, fiber optic cable is recommended. Due to its immunity to lightning, fiber optic cable should also be used for outdoor cable runs (rated for outdoor use).

♦ Where will the equipment be located?

Will switches be mounted in racks, in cabinets, or placed on shelves or tabletops? Take into consideration thermal requirements, especially if the equipment will be mounted in an enclosed cabinet or area. High or moderate electromagnetic noise sources, for example, machinery, switchgear, high-voltage lines, and so forth, in close proximity must be avoided to ensure reliable operation.

♦ Have plans for future expansion been considered?

25


Consider installing Ethernet switches with additional ports so that control stations or workstations can be easily added in the future. Running extra copper or fiber optic cable will allow for network expansion later.

♦ Have maintenance and troubleshooting provisions been made?

Leaving at least one unused port on each Ethernet switch allows for maintenance and troubleshooting of managed switches. A system network management pack-age (SNMP) must be available that runs on a separate PC, not an I/A Series work-station. Patch panels at the end of cable segments facilitate troubleshooting and network re-configuration.

♦ Newer switches use 1 Gb uplinks. Can a new MESH network be built including legacy 100 Mb uplinks?

♦ While a MESH network may be built using 100 Mb uplinks, this is not recom-mended due to loading concerns. Note that Security Enhanced Configurations require the use of 1 Gb uplink ports.

♦ Are ample ac power outlets available at the proper location:

♦ For the Chassis switch?

♦ For switches?

♦ For redundant power supplies?

♦ For media converters and ac to dc power adapters for media converters?

Traffic ConsiderationsSpecific traffic requirements should be considered when planning The MESH control network.

♦ Should certain control stations or workstations be connected to the same switch?

To reduce traffic through the root switches, it may be advantageous to group con-trol stations or workstations according to department, process, or other criteria that is important to the site or organization.

♦ Should certain control stations or workstations not be connected to the same switch?

It is recommended that each control station or workstation from a particular group be connected to two separate switches. If one switch were to fail, these con-trol stations or workstations can access the network through the redundant switch. Without a redundant network, none of the workstations from that group would have network access. Workstations can have two Ethernet ports to access the network.

Network LoadingUnderstanding the details of the network traffic flow is an important part of planning and imple-menting The MESH control network for an I/A Series system. This provides insight on how to verify that there is sufficient network bandwidth available between network devices. This is diffi-cult to measure, since available bandwidth is a dynamic quantity where the amount of traffic that can be transmitted over a link can change given its current traffic conditions and what applica-tions are running on those devices. The process of designing industrial networks is a very chal-lenging task due to its inherent complexity. A load prediction can be achieved by approximating the environment, modeling the network components and analyzing the interrelations. This pro-

26


cess works well for industrial networks since they tend to be more static and are typically slow to evolve.

It is recommended that The MESH control network steady state load not exceed a maximum load of 50% of any given link.

The MESH control network is a full-duplex, point-to-point system. This network does not have a collision domain. Unlike half-duplex links, full-duplex links do not deteriorate as the network load increases. All end-devices can transmit at will. However, if The MESH network is designed with 100 Mb uplinks communication from these end-devices, this could cause a bottleneck due to insufficient available network bandwidth1. Since The MESH network runs in full duplex mode, both transmit and receive channels can run at 100 Mb simultaneously without degradation on the adjacent channel.

The MESH network traffic rates are also affected by third-party applications or user applications that generate high packet rates. Workstation to workstation operations on The MESH network, such as copying extremely large files, can also result in a temporary high bandwidth usage up to 50% of the network:

Network loading will be represented by the percent of time that the network is in use over a given period. By definition, individual Ethernet segments can only transmit one packet at a time. For any given moment, the Ethernet segment is either at 100% loaded (transmitting a packet), or at 0% utilization (idle). The network loading percentage shows the percentage of time the network is in use over a set period.

When calculating the network loading, you need to know how many bytes of network traffic are being handled by the network over a set period. This involves totaling the input (or output) byte counts for a set period, and dividing by the total capacity of the device interface for that period. To determine the total number of bits received on the interfaces, each of the packet byte rates is multiplied by 8.

%loading = (dataSent * 8) / (intSpeed * sampleTime) * 100

Where:

dataSent = total number of bytes sent

intSpeed = maximum connection speed

sampleTime = duration of time required for transmissions

For example, for a system having 6 x 1500 byte sends on 100Mb connection at maximum of 2Mb (2 * 1,000,000 bps) every 500 milliseconds:

7.2% = (6 * 1500) * 8 / (2,000,000 bps * 0.50secs) * 100

Equipment ConsiderationsThe MESH control network was designed and tested for operation with the Ethernet switches listed in “Reference Documents” on page xii. The network may operate with similar, off-the-shelf

1. Flow control and rate limiting are disabled, and full-duplex cables are used, allowing the controllers transmitted packet to egress though the network without delaying or filtering. However, by doing this, a packet on a heavily loaded network can cause traffic to slow. When using these features, the switch acts as a traffic cop controlling the data to maximize the traffic flow. Disabling these features allows the traffic to flow faster but can cause issues when traffic loads are high.

27


equipment, but Invensys Systems, Inc. is not responsible for any system malfunctions that may occur if such equipment is used.

If you use your own network equipment:

1. You must meet the bandwidth requirements (1 Gb for uplink ports and 100 Mb for ports) for the I/A Series equipment you have chosen.

2. There can be no Layer 3 inter-network devices (for example, routers) between any I/A Series equipment.

Hardware RequirementsWhile switches with 100 Mb uplink ports are allowed on networks with standard configurations, Security Enhanced Configurations require the use of 1 Gb uplink ports. However, 1 Gb uplink connections should be used to interconnect switches in all configurations, if available.

Switch Utilization in the Standard ConfigurationsAny switches defined in the table “Invensys-Supplied Ethernet Switches” in the “Introduction” of The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA) can be used in Standard Configurations.

Note that C-Series switches (P0973BL/HA) are used only in a Standard Configuration and can-not be used in the Security Enhanced Configuration.

Switch Utilization in the Security Enhanced ConfigurationsRefer to the Appendix table “Qualified Switch Standard/Security Enhanced Configuration Com-patibilities Matrix” in The MESH Control Network Operation, and Switch Installation and Config-uration Guide (B0700CA) for a list of allowed switch types in Security Enhanced Configurations.

Since the Security Enhanced Configurations deploy Loop Detection Policy algorithm (LDP) methods to monitor and protect the network from RSTP or Data Loop (storm) failures, specific switch types must be used in a variety of required locations within the network. The table listed above identifies the switch types which fall into the categories “Security Enhanced” and “LDP Deployable”.

♦ In a linear topology in a Security Enhanced Configuration, both switches must be from the “LDP Deployable” category.

♦ A ring topology cannot be supported as Security Enhanced Configuration.

♦ In a star topology in a Security Enhanced Configuration, both root and backup root switches must be from the “LDP Deployable” category. All other switches within the star network can be from the “Security Enhanced” category.

♦ In an inverted tree topology in a Security Enhanced Configuration, all switches within the network must be from the “LDP Deployable” category.

♦ The modified inverted tree topology in a Security Enhanced Configuration requires that all switches within the network core are “LDP Deployable”. All outer edge switches within the tree network can be any switch type; however, the redundant switches (A/B switches) must be deployed on different tiers.

28


Firmware ConsiderationsWhen planning which Invensys-supplied Ethernet switches to use when building The MESH control network for your system, be aware that certain versions of these switch’s firmware may be incompatible with one another.

Refer to the Appendix “Qualified Switch Firmware Compatibilities Matrix” in The MESH Con-trol Network Operation, and Switch Installation and Configuration Guide (B0700CA) for the I/A Series qualified and compatible firmware for each switch.

The MESH Control Network Design RulesWhen designing The MESH control network, the following rules/guidelines should be kept in mind.

Standard Configuration Design RulesThe following rules apply to the standard design of The MESH control network:

1. No more than 1920 logical stations where:

♦ Fault-tolerant stations count as one logical station

♦ Workstations count as one logical station

♦ Ethernet switches count as one logical station.

♦ FCMs are not counted.

2. No more than 10,000 IP addresses where:

♦ A fault-tolerant FCP270 counts as one address

♦ A fault-tolerant ZCP270 counts as two addresses

♦ An ATS station counts as two addresses

♦ MESH workstations count as two addresses prior to I/A Series system version 8.2; MESH workstations with I/A Series system V8.2 or later count as one address.

♦ Nodebus workstations count as one address

♦ Ethernet switches count as one address

♦ FCMs count as one address.

3. No more than 250 managed switches are allowed.

4. There should be no more than seven switches in the path between any two devices in The MESH control network.

5. Routers are not allowed in The MESH control network.

6. Switch-to-Switch connections (uplink ports) should be made using 1 Gb uplink ports to allow enough bandwidth for network traffic of I/A Series equipment

7. Only two configured root bridges are allowed in the network.

8. Horizontal switch interlinks (links between switches of the same tier) are not allowed, except between the root and backup root switches. (See Figure 2-1 on page 33).

9. The MESH control network must have redundant uplink connections between the root and backup root switches.

29

saravana.prabu

Note

need to check


10. The MESH control network will support up to six VLANs, one (1) of which must be reserved for I/A Series system devices (VLAN 2 “I/A Control Ports”).

a. Only one I/A Series control system can be supported on The MESH control net-work.

b. If VLANs are deployed, all I/A Series control devices shall be attached to VLAN 2 FE ports.

c. If VLANs are deployed, all switches in the network must have VLAN 2 enabled on the switch’s uplink ports.

d. VLANs other than VLAN 1 and 2 shall not utilize protocols other than TCP/IP and/or UDP/IP.

e. No VLAN port shall exceed 50% loading

f. No end devices with transfer (Tx) and receive (Rx) speeds greater than 100Mb are supported.

NOTEIt is recommended that one port on each managed Ethernet switch be reserved for testing and diagnostic purposes. No device should be connected to this port.

Non I/A Series EquipmentThe following rules apply to non I/A Series equipment:

1. Non I/A Series equipment (for example, routers, hubs and unmanaged switches) are not allowed to be connected to The MESH control network.

2. A corporate or plant enterprise network should not be connected directly to The MESH control network. The recommended way to communicate from a corporate or plant enterprise network is by using an additional NIC on an I/A Series workstation.

Security Enhanced Configuration Design RulesWhen designing the Security Enhanced Configuration, the following rules/guidelines must be followed.

1. No more than 1920 logical stations where:

♦ Fault-tolerant stations count as one logical station

♦ Workstations count as one logical station

♦ Ethernet switches count as one logical station.

♦ FCMs are not counted.

2. No more than 10,000 IP addresses where:

♦ A fault-tolerant FCP270 counts as one address

♦ A fault-tolerant ZCP270 counts as two addresses

♦ An ATS station counts as two addresses

♦ MESH workstations count as two addresses prior to I/A Series system version 8.2; MESH workstations with I/A Series system V8.2 or later count as one address.

30


♦ Nodebus workstations count as one address

♦ Ethernet switches count as one address

♦ FCMs count as one address.

3. No more than 250 managed switches are allowed.

4. There should be no more than seven switches in the path between any two devices in The MESH control network.

5. Routers are not allowed in The MESH control network.

6. Switch-to-Switch connections (uplink ports) must be made using 1 Gb uplink ports to allow enough bandwidth for network traffic of I/A Series equipment.

7. Core switches must be only LDP-supported switches, as described in the Appendix table “Qualified Switch Standard/Security Enhanced Configuration Compatibilities Matrix” in B0700CA.

8. The root switch must have the lowest IP address in the network, or at least an address lower than the backup root switch.

9. Only two configured root bridges are allowed in the network.

10. Do not use an application on the network which requires IGMP, such as IP video.

11. Horizontal switch interlinks (links between switches of the same tier) are not allowed, except between the root and backup root switches. (See Figure 2-1).

12. The Circuit Breaker Policy (CBP) should be deployed at all uplinks ports facing the outer edge (away from the root), ports that are not and will not be blocked by span-ning tree.

13. The Beacon Priority Policy (BPP) should be deployed at all uplink ports facing towards the root.

14. The Backplane Circuit Breaker Policy (BPCB) (used only for E-series bridge cards P0973BS) should be used to detect RSTP failures on the E-series second and third generation blades only (defined in the Appendix table “Qualified Switch Stan-dard/Security Enhanced Configuration Compatibilities Matrix” in B0700CA). It is not recommended that any other switches be interfaced to the second and third gener-ation blades via uplink or data ports. Blades protected with the BPCB policy (i.e. E-series second and third generation blades) should reside at the outer most edge of the network. A detailed description of BPCB and other policies can be found in the appendix section “Deploying Loop Detection Policies” in the B0700CA document.

15. All edge switch uplink ports facing the root require the spanning tree admin path cost to be increased (≥200000), this ensures efficient port blocking at the edge.

16. Methods for active network monitoring (e.g. NetSight Policy Manager) are not required but are recommended for medium to large networks.

17. The MESH control network must have redundant uplink connections between the root and backup root switches.

18. The MESH control network will support up to six VLANs, one (1) of which must be reserved for I/A Series system devices (VLAN 2 “I/A Control Ports”).

a. Only one I/A Series control system can be supported on The MESH control net-work.

31


b. If VLANs are deployed, all I/A Series control devices shall be attached to VLAN 2 FE ports.

c. If VLANs are deployed, all switches in the network must have VLAN 2 enabled on the switch’s uplink ports.

d. VLANs other than VLAN 1 and 2 shall not utilize protocols other than TCP/IP and/or UDP/IP.

e. No VLAN port shall exceed 50% loading

f. No end devices with transfer (Tx) and receive (Rx) speeds greater than 100Mb are supported.

32


Figure 2-1. Large Network (Security Enhanced Configuration)

Uplink paths shouldbe gigabit paths

Interlink flows across the network, i.e. on the same tier, this is only recommended at the root tierwith Circuit Breaker PBQ and SBQ deployed.

Interlinks should flowtowards or away from root,i.e. from one tier to the other.

Note: On N-series switches, the two interlinks between the root and backup should be onseparate blades.

33


Additional Guidelines for Planning Security Enhanced ConfigurationThe design of each instance of The MESH control network is different, and depends on the needs and requirements of the individual site. This section provides the information necessary to help network designers to plan The MESH control network that meets the needs of their specific site.

The MESH Control Network with LDP (also referred to as the Security Enhanced Configura-tion) isolates storming switches, and allows communications to continue on The MESH. In a properly constructed network, all switches on the network are able to communicate with each other with a high degree of reliability and redundancy. The MESH control network is con-structed so that there are redundant links that form physical loops in the network which are con-trolled (Blocked) by Rapid Spanning Tree Protocol (RSTP), creating a logical loop-free network. In the event of a failure of this function the loop detection algorithm provides a backup to this type of failure.

There are rules that must be followed when connecting switches to form The MESH control net-work. The following section explains the dynamics affecting the physical topology and the traffic considerations of The MESH control network.

Security Enhanced Configuration Design ConsiderationsOnce the general plan for the Security Enhanced Configuration has been outlined, the physical design of the network can be defined. Refer to B0700CA for the rules concerning network config-uration/topology and list the specifications for the network devices (switches, converters, interface modules) and cabling, and to choose the equipment that is needed to implement the network plan.

NOTEFor a variety of questions to help you define both the Standard and Security Enhanced Configurations, refer to “Network Considerations” on page 23.

When the physical design has been finalized, make a drawing or map of the network topology and save it. The map should be updated whenever a physical change is made to The MESH control network.

Spanning Tree Behavior

A Spanning Tree design must be implemented correctly to provide a loop-free network during normal operations. When LDP is deployed, improper configuration can cause “false” triggers of the policy which will disable active ports/switches.

Users should ensure that the edge switches keep their redundant connections blocked. The path cost typically involves the sum of the path costs, that is, of the links that are traversed. This cost can be adjusted by altering the default assigned path costs for individual links. An edge switch may not block its ports in an expected order, though other switches may do so. Edge switches must have their path cost administratively increased to get their local ports to block. This keeps the “Beacon” packet from being forwarded to a port with the “Circuit Breaker” policy. Increasing a path cost on a switch port, increases its likeliness to block all local ports on edge devices in a MESH topology. This path cost is dependent on the switch interconnections between the switch tiers. The edge switch uplink ports path cost must be increased to ≥200000. Ports designated as the “Beacon” ports, must have their path cost set from default value of 0 to a value ≥400000.

34


The admin path cost is automatically adjusted when utilizing the I/A Series Switch Configurator Application Software (Rev 1.1.4 or greater) discussed in The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA). However, if other methods are used, refer to the switch configuration documentation provided with the switch to perform this func-tion.

IGMP “Beacon” Behavior

LDP functions utilize IGMP packets which act as a beacon. The IGMP “Beacon” is enabled only on the root and backup root switches in the network when deploying LDP. Only the designated root switch with the lowest IP address will send out a “Beacon” query packet. The Secondary “Beacon” queryer (SBQ) acts as a backup and only sends out a “Beacon” query packet if the root fails. The root switch must be configured with the lower IP address of the root and backup root pair in order for LDP to operate properly. The switch with the lowest IP address and with its IGMP “Beacon” enabled will be chosen as the Primary Beacon Queryer (PBQ). When configur-ing the network in SysDef or IACC System Editor, the root switch should be designated with the lowest switch IP address of the root/backup root pair.

In the event of a root switch failure the SBQ on the backup root will take over. Once the root switch has been placed back online PBQ from the root will send out an IGMP “Beacon” packet stopping the SBQ on the backup root from transmitting its Beacon packet.

CAUTION!When using Netsight Policy Manager to manage a Security Enhanced Configura-tion switch, you must have Policy Manager 2.2.0 build 17 or greater installed. In the event of a root switch failure, the LDP Beacon “PBQ” will become disabled and the functions of the root switch will move to the backup root switch along with the LDP Beacon “SBQ”. Once the root switch failure has been resolved and the switch has been placed back on the network, the LDP Beacon “PBQ” will become enabled, disabling the “SBQ”. Due to this event, the redundant root switch links between the root and backup root will be viewed by LDP as a loop within the network, which causes LDP to disable one of the links (the blocking port). To recover from this rule hit, the root switch (switch with the lowest IP address) needs to regain all root functions. Netsight Policy Manager 2.2.0 build 17 or greater has the correct software required to recover from this rule hit. This event can be prevented if VLAN 2 is deployed on the network, by moving the root switch host ports to a secure VLAN “VLAN 2” (host port moved to VLAN 2 is required for an I/A Series system) will resolve the false port hits between the two root switches.

The default timing for the IGMP protocol causes one packet to be sent every 125 seconds. In order to provide a “Beacon” packet that will allow a loop to be detected within one second the IGMP timer is set to one second for LDP. Since IGMP packets are multicast packets they are for-warded out from the root to the edge switches of the network. In the event of a loop, the packet will be forwarded back towards the root where a port with a circuit breaker policy will detect it and disable the port shutting down the loop.

Normally IGMP packets are used to sustain and prune multicast flows in a network and are used for IP video. Any application which uses IGMP, such as IP video, cannot be used on a Security Enhanced MESH Control Network since it will disrupt LDP operation.

35


The MESH Control Network DesignOnce the general plan for The MESH control network has been outlined, the physical design of the network can be defined. The following two sections describe the rules concerning network configuration/topology and list the specifications for the network devices (switches, converters, interface modules) and cabling. Use these sections to choose the equipment that is needed to implement the plan that was derived from the answers to the questions at the beginning of this chapter. When the physical design has been finalized, make a drawing or map of the network topology and save it. The map should be updated whenever a physical change is made to The MESH control network.

I/O Network Design RulesThe following rules apply to the I/O network design of The MESH control network:

1. Only Invensys-qualified switches supported for The MESH control network are per-mitted on the I/O network.

2. The I/O network does not support redundant ISL links between switches. Only single links between switches are allowed.

3. Routers are not allowed in the I/O network.

4. Switch-to-Switch connections (ISL uplink ports) must be made using 1 Gb uplink ports to allow enough bandwidth for network traffic of the I/A Series equipment. 100 Mbps uplinks are not supported on the I/O network.

5. Configured root bridges are allowed in the I/O network, but not required.

6. The “A” Fieldbus switches must not be interlinked to the “B” Fieldbus switches (see Figure 1-7 on page 14). The two Fieldbuses must maintain separation.

7. Only the linear topology is supported on the I/O network, with no more than seven switches in series. It is recommended to use the fewest number of switches with which you can configure the I/O network.

8. The same number of switches must be maintained on both the “A” side and the “B” side of the I/O network.

NOTEA dedicated I/O network switch can be configured with the Switch Configurator Application Software (SCAS), as discussed in The MESH Control Network Opera-tion, and Switch Installation and Configuration Guide (B0700CA).

NOTEIt is recommended that one port on each managed Ethernet switch be reserved for testing and diagnostic purposes. No device should be connected to this port.

The MESH Control Network TopologiesA key feature of The MESH control network is that single points of failure will not prevent com-munications between all the devices in the network. This is accomplished by using a MESH net-work design in which each I/A Series station is ideally connected to two different Ethernet

36


switches. Each of the Ethernet switches is then connected to two other Ethernet switches. This design provides redundant data paths so that the failure of a single device doesn’t cause opera-tional network problems.

Security Enhanced Configurations provide the loop detection algorithm known as LDP to pro-tect against RSTP and/or data loop (storm) failures. When deployed on a Security Enhanced Configuration, LDP allows for the isolation of a defective switch or port (depending on the type of failure) during a network storm when caused by a spanning tree failure or a data looping fail-ure. This function will prune the defective loops, maintaining network communications between devices during a network storm.

NOTEIf the network is constructed with less than two connections between devices, data traffic could be disrupted if any single device fails.

The MESH control network can be constructed using two network configuration/topology meth-ods. The first is the Standard Configuration as described in this section under “Standard Config-urations” below and the second is the Security Enhanced Configuration as described in “Security Enhanced Configurations” on page 52. The topology chosen depends largely on the size, com-plexity, site requirements, and network specifications of The MESH control network. The follow-ing paragraphs provide some guidelines for choosing a network configuration (topology).

NOTEAll graphics of switches and media converters below are intended as generic illustra-tions of networking concepts and do not necessarily reflect the currently offered products.

Standard ConfigurationsThere are five basic standard configurations that are supported by The MESH control network. As well, the variations of these topologies listed below can be made as long as all design rules are met. Be aware that some restrictions to these variations listed below may be warranted.

The basic standard configurations and their approved variations are as follows:

♦ Linear

♦ Ring

♦ Star (Variation: Double Star)

♦ Inverted tree

♦ Modified inverted tree (Variation: Inverted Stepped Tier Tree)

Standard Small Networks

A small network, consisting of two switches, could be configured as in Figure 2-2.

37


Figure 2-2. Small Network (Standard Configuration)

In this example, two fiber optic Ethernet switches are used and the workstation and controller have a 100 Mb connection to each of the switches. The connections between the switches can use the switch’s high speed uplink ports (1 Gb).

NOTETwo connections between switches are required for proper redundancy.

Larger Chassis switches can be used instead of the non-Chassis Fiber switches. Larger Chassis switches will allow hundreds of workstations, controllers and FCMs to be connected to The MESH control network. This provides a small network but quite a large control system. Multiple FCP270/FCM100s/ZCP270s, ATS modules and workstations can be connected to a redundant switch.

Gigabit Ethernet overFiber Uplink Ports

FCP270

P92

Fiber ManagedSwitches

38


Standard Medium Networks

A medium-sized network of three to seven switches can be configured as a ring. Figure 2-3 shows an example of a ring topology containing six switches. In this example, all the switches are fiber optic and each switch is connected to the two switches adjacent to it. If one of the switches should fail, the ring is broken but RSTP (802.1w) reroutes data around the break and the network remains operable. Typically a failover time of less than 1 second is achievable using switches qual-ified and supplied by Invensys and configured in accordance with I/A Series documentation.

NOTEThe I/A Series system imposes a seven switch limit on the length of any data path. This means that there can be no more than seven switches between any two devices in the network.

Figure 2-3. Medium Network (Ring) (Standard Configuration)

A medium and large-size network can also be configured in a star topology as shown in Figure 2-4. The star topology is the preferred topology for control systems. In star topology, two switches make up the backbone of the network and should be configured as the root and backup root switches. The other switches “edge switches” in the network are connected to both root switches. If desired, other devices (workstations or controllers) can be connected directly to the root switches.

A medium network Star Configuration has a limitation of 40 switches on the edges when Gold Series blades are used and 166 when Platinum Series Blades are used. This limit is derived from the maximum number of Gb ports on the Chassis switch.

Two ports are used for interconnection between the primary and backup root.


Primary Root

Backup RootFiber ManagedSwitches

39


Figure 2-4. Medium or Large Network (Star) (Standard Configuration)

The Double Star Configuration (a variation of the Star Configuration, shown in Figure 2-5) allows the user to benefit from the Star Configuration advantages while allowing the user to deploy a much larger sized network. Bandwidth considerations between the Star topology switches must be observed, to minimize bandwidth consumption. The majority of peer-to-peer end device communications should reside local to the individual Star topologies.


Primary

BackupRoot

Root

ExpandableChassis Switch



40


Figure 2-5. Medium or Large Network (Double Star) (Standard Configuration)

Standard Large Networks

Large networks should be designed to use an inverted tree topology or a modified inverted tree topology.

For the inverted tree topology, Ethernet switches are arranged in tiers, with primary and backup root switches in the top tier. Each switch is connected to two different switches in the tier above it and end stations may be connected to any of the switches. A large network containing 12 switches in four tiers is illustrated in Figure 2-6.

NOTEThere is a limit of four tiers on both inverted tree topology or a modified inverted tree topology. This is to prevent the path between any two devices from containing more than seven switches.


N-Series N-SeriesSecondary RootPrimary Root

Expandable Chassis Switches

Tier 1

Tier 2

Tier 3

A-, I- or V-Series Switches A-, I- or V-Series Switches

A-, I- or V-Series Switches



C-Series Switches

41


Figure 2-6. Large Network - All Blades (Standard Configuration)

Uplink paths shouldbe gigabit paths

42


The Inverted Stepped Tier tree topology, shown in Figure 2-7, is an example which demonstrates that users can utilize a non-symmetrical tree topology.

In this topology, switches connected between every other tier cannot be separated by more than 100 meters due to the distance restrictions on the copper uplinks. Every other tier separation can be greater than the 100 meter restriction, by using fiber Mini-GBICs. Separation greater than the 100 meter restriction can be between any Tier, but EMI noise should be taken in consideration when using copper uplink ports.

In the example in Figure 2-7, the uplinks between tiers 2 and 3 are not required to be interfaced to the switch directly above each. However, all switches in the network must be interfaced with two (2) uplinks to the tier above them.

When using I-series switches at the out edge, fiber Mini-GBICs must be used to interface the switches into The MESH control network.

If 100Mb uplinks are used instead of 1Gb uplinks, bandwidth consumption must be minimized between the two tiers. To minimize bandwidth consumption as much as possible, the end device peer-to-peer communications should reside local to the switch to which they are connected.

43


Figure 2-7. Large Network - Inverted Stepped Tier Tree - All Blades (Standard Configuration)

CAUTION!In this topology, switches connected between every other tier cannot be separated by more than 100 meters due to the distance restrictions on the copper uplinks. Every other tier separation can be greater than the 100 meter restriction by using fiber Mini-GBICs. Separation greater than the 100 meter restriction can be between any tier, but EMI noise should be taken in consideration when using cop-per uplink ports.




Tier 1

Tier 2

Tier 3

A-Series


Switches

Tier 4



A-SeriesSwitches

A-SeriesSwitches

44


The modified inverted tree topology is similar to the inverted tree topology but with consider-ations to cost restraints. This topology utilizes low end, low cost switches as edge devices. In this topology, the switches are arranged in tiers, with the root switches in the top tier and up to three tiers below them (maximum of four tiers). The two root switches are connected to each other and the other switches each have a connection to two of the switches in the tiers above and below them. Low-end low-cost switches must be added to the outer edge at different tiers. Redundant data paths allow the network to continue to operate if any one component fails.

A large network containing 10 switches in four tiers is illustrated in Figure 2-8.

There can be no more than four-tiers of switches (including the root) in order to comply with the I/A Series system requirement, limiting the number of switches between devices to seven.

Figure 2-8. Large Network - Blades and Low-Cost Switches (Standard Configuration)

45


When designing the physical layout of a large network, the following guidelines apply:

♦ Switch-to-Switch connections (uplink ports) should be made using 1 Gb uplink ports to allow enough bandwidth for network traffic of I/A Series equipment

♦ There should be a primary and backup root switch on the network

♦ There should be no more than four tiers of switches, including the primary and backup root switches

♦ There should be no horizontal connections between switches on the same tier except the root and backup.

♦ This minimizes the number of switch interconnections in order to facilitate faster network respawning of the network tree.

♦ This also reduces the likelihood of a loop occurring.

♦ Each tier should contain an even number of switches

♦ Each switch should be connected to two different switches in the tier above it

♦ There should be two connections between the primary root and the backup root.

Example:

If it has been determined that the number of workstations, controllers and FCMs that must be connected requires a network containing ten switches, the total number of switches needed will be 12 (ten switches plus primary and backup root switches). Refer to Figure 2-6 for a diagram of the finished network.

46


Step 1: Determine Physical Structure

The primary and backup root switches will be in the top tier, and the remaining ten switches will be divided between the other three tiers. To determine the number of switches in the three tiers, divide the ten switches by three. The result is three (with one left over), which suggests that there should be approximately three switches per tier. Keeping in mind that there should be an even number of switches in each tier, switches are added or subtracted until this guideline is met. In this case we arrive at a solution that has two switches in the second tier, and four switches in each of the remaining two tiers.

Figure 2-9. The MESH Control Network Tiers

Switch Switch

Switch Switch

Switch Switch

Switch Switch

Switch SwitchSwitch Switch

BackupPrimary

Root Switches

2nd Tier

3rd Tier

4th Tier

47


Step 2: Connect Root Switches

Two connections are made between the primary and backup root switches. These connections should be from gigabit uplink port to gigabit uplink port. Switch-to-Switch connections (uplink ports) should be made using 1 Gb uplink ports to allow enough bandwidth for network traffic of I/A Series equipment. Connecting a gigabit uplink port to a gigabit uplink port increases the speed of the response.

Figure 2-10. Root Switches Connected

Switch Switch

Switch Switch

Switch Switch

Switch Switch


BackupPrimary

Root Switches

2nd Tier

3rd Tier

4th Tier

48


Step 3: Connect Second Tier

Once the primary and backup root switches have been connected, separate connections should be made from each second tier switch to both the primary and backup root switches.

Figure 2-11. Second Tier Connections

Switch Switch

Switch Switch

Switch Switch

Switch Switch


BackupPrimary

Root Switches

2nd Tier

3rd Tier

4th Tier

49


Step 4: Connect Third Tier

Each switch on the third tier should have separate connections to two different switches in the tier above it. In this case there are only two switches in the tier above, so a connection is made to each of them.

Figure 2-12. Third Tier Connections

Step 5: Connect Fourth Tier

The final step is to connect each switch in the forth tier to two different switches in the tier above it. Connections may be made to any two switches in the tier, but consideration should be made with respect to keeping a manageable traffic load on each switch.

Figure 2-13. Fourth Tier Connections

Switch Switch

Switch Switch

Switch Switch

Switch Switch


BackupPrimary

Root Switches

2nd Tier

3rd Tier

4th Tier

Switch Switch

Switch Switch

Switch Switch

Switch Switch


BackupPrimary

Root Switches

2nd Tier

3rd Tier

4th Tier

50


Step 6: Label the Switches

Switches and cables should be labeled to aid in organizing the network connections. The strategy illustrated here is one method. This example uses the ‘A’ and ‘B’ labels as an aid in migrating from two separate nodebus networks to The MESH control network. Numeric superscripts are added to designate the tier and the switch number within the tier. Any labeling strategy which helps to keep the network connections organized may be used.

Figure 2-14. Labeling the Switches

Switch Switch

Switch Switch

Switch Switch

Switch Switch


BackupPrimary

Root Switches

2nd Tier

3rd Tier

4th Tier

A11

A21

A31 A32

A41 A42

B11

B21

B31 B32

B41 B42

51


Security Enhanced ConfigurationsThere are four basic Security Enhanced Configurations that are supported by The MESH control network. These are:

♦ Linear

♦ Star (Variation: Double Star)

♦ Inverted tree


The following diagrams provide examples of the different topologies deploying the loop detection algorithm (LDP) as well as recommendations on where they might be used. All Security Enhanced Configurations require specific switch models as the root or backup root. The inverted tree requires switches which have LDP. The modified inverted tree requires all switches except the outer edge switches to have LDP.

Refer to the “LDP Deployable” switches in the Appendix table “Qualified Switch Standard/Secu-rity Enhanced Configuration Compatibilities Matrix” in The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA) for the applicable switch models.

“Linear” Networks (Security Enhanced)

A small network, consisting of two switches, could be configured as shown in Figure 2-15.

Figure 2-15. Small Network (Security Enhanced Configuration)

NOTETwo connections between switches are required for proper redundancy.

Larger Chassis switches can be used instead of the non-Chassis Fiber switches. Larger Chassis switches will allow hundreds of workstations, controllers and FCMs to be connected to The MESH control network. This provides a small network but quite a large control system. Multiple FCP270/FCM100s/ZCP270s, ATS modules and workstations can be connected to a redundant switch.


FCP270

P92

DFE PlatinumSwitches

52


Star Topology (Security Enhanced)

The star topology is the preferred topology for all networks. It is the easiest to maintain, expand, and deploy. In a star topology, the switches at the outside edge of the network have connections to each of the two root switches. The two root switches are connected to each other and all edge switches. Redundant data paths allow the network to continue to operate if any one component fails.

Figure 2-16 illustrates a star network in the Security Enhanced Configuration. In a Security Enhanced star topology, as many as 166 edge switches can be connected to the Chassis switch using 1 Gb uplinks. In this configuration, if an edge switch spanning tree protocol fails, LDP will remove the loop that was created by the failure. In the event the switch fails in a manner that causes a network flood, LDP will remove the defective switch from the network, allowing its redundant switch to take over. An inverted tree topology or modified inverted tree topology can be considered if there is a larger number of edge switches required.

Figure 2-16. Star Topology (Security Enhanced Configuration)

53


The double star topology allows the user to benefit from the star topology advantages while allow-ing the user to deploy a much larger sized network.

Bandwidth considerations between the star topology switches must be observed. To minimize bandwidth consumption, the majority of peer-to-peer end device communications should reside local to the individual star topologies.

Figure 2-17. Double Star Topology (Security Enhanced Configuration)




Tier 1

Tier 2

Tier 3

A-, I- or V-Series SwitchesA-, I- or V-Series Switches

N-SeriesChassis

A-, I- or V-Series SwitchesA-, I- or V-Series Switches


Switch

N-SeriesChassisSwitch

54


Inverted Tree Topology (Security Enhanced)

The inverted tree topology is suited for very large networks with specific physical constraints. In this topology the switches are arranged in tiers, with the root switches in the top tier and up to three tiers below them (maximum of four tiers). The two root switches are connected to each other and the other switches each have a connection to two of the switches in the tiers above and below them. Redundant data paths allow the network to continue to operate if any one compo-nent fails. The deployment of the loop detection algorithm allows for redundant network condi-tion monitoring by RSTP and LDP.

NOTEWhen utilizing an Inverted Tree Topology in the Security Enhanced Configura-tion, all switches within the network must be switch types that support the loop detection algorithm.

An inverted tree network topology is illustrated in Figure 2-18.

There can be no more than four-tiers of switches (including the root) in order to comply with the RSTP requirement, limiting the number of switches between devices to seven.

NOTEThe Stepped-Tier Tree variation example of this topology is not shown as a Security Enhanced Configuration. However, this configuration can be utilized with the following rule heeded:All switches in the network must be interfaced with two (2) uplinks to the tier above it, all uplinks attached to a tier above it must be attached to a switch that is “LDP deployable” and if a switch is attached to a tier below it, it too must be “LDP deployable”.Refer to the “LDP Deployable” switches in the Appendix table “Qualified Switch Standard/Security Enhanced Configuration Compatibilities Matrix” in The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA) for the applicable switch models.

55


Figure 2-18. Inverted Tree Topology (Security Enhanced Configuration)

56


Modified Inverted Tree Topology (Security Enhanced)

The modified inverted tree topology is similar to the inverted tree topology but with consider-ations to cost restraints. The modified inverted tree topology is suited for very large networks. However, this topology utilizes low end, low cost switches as edge devices. In this topology, the switches are arranged in tiers, with the root switches in the top tier and up to three tiers below them (maximum of four tiers). The two root switches are connected to each other and the other switches each have a connection to two of the switches in the tiers above and below them. Low-end low-cost switches can be added to the outer edge at different tiers. Redundant data paths allow the network to continue to operate if any one component fails.

By deploying this, the loop detection algorithm allows for redundant network condition monitor-ing by RSTP and LDP.

An inverted tree network topology is illustrated in Figure 2-19.

There can be no more than four-tiers of switches (including the root) in order to comply with the I/A Series system requirement, limiting the number of switches between devices to seven.

57


Figure 2-19. Modified Inverted Tree Topology (Security Enhanced Configuration)

I/O Network Topology ConfigurationsOnly linear topologies are supported for use in a dedicated I/O network.

The Linear I/O Network

The linear I/O network can consist of a single A bus switch and a single B bus switch (two switches total) or up to seven switches interlinked together via 1Gb (ISL) uplinks per bus. Addi-tional switches should only be used when the total FCM and ZCP port count exceeds the switch port count, or when the geographical conditions dictate the requirement for more than one switch. When using Single Mode Fiber ISLs, the distance between two switches can be up to, but not exceed, 80 km. When using Multi-Mode Fiber ISLs, the distance can be up to 2 km. No

58


crossover ISL links between the “A” and “B” buses are supported on the linear I/O network. Refer to Figure 1-7 on page 14.

Switch and Fiber Cable Budget and Loss

Fiber Cable Budget Cable and LossThe typical power budget for 100FX system of The MESH control network with 62.5/125 m fiber is 11.0 db and worst case (includes aging) is 10.0 db.

Table 2-1 lists some typical and worst case power losses for fiber optic cable, splitter/combiners and patch cords.

Fiber Optic BudgetsRefer to the documentation included with your Invensys qualified Ethernet switches for details of their fiber optic budgets.

The MESH Control Network CablingThe following sections provide the specifications and transmission distance capabilities for the fiber optic, and twisted-pair cabling used in The MESH control network.

Switch and Converter Fiber Optic CablingTwo types of fiber optic cable can be used to connect devices in The MESH control network: multimode fiber optic cable and single mode fiber optic cable. Each cable type is used in different applications and the devices that are connected must support the type of cable being used. The following sections give the supported uses and maximum transmission distances for each type of fiber optic cable.

Table 2-1. Fiber Cable Power Losses

Link Element Value Comments

Splitter/Combiner 4.5 db

MT-RJ Patch Cord 0.5 db 0.75 db worst case

LC, SC, and other similar patch cords

0.15 db 0.5 db worst case

Singlemode fiber cable (SMF)

0.5 db/km at 1300 nm wavelength

0.275 dB at 1550 nm wavelength

Multimode fiber cable (MMF)



Splice loss 0.25 dB Mechanical splice

50/125 µM mating 62.5/125 µM loss

5.25 dB -5.0 dB mating loss and a -0.25 dB for mechanical splice

Aging reserve 1.0 dB 62.5/125 micron cable

Patch Panel loss 2.0 dB average

59


Multimode Fiber Optic Cabling

Multimode fiber optic cable is used to connect stations to switches, media converters to other devices, and, with the appropriate uplink module, Ethernet switches to other Ethernet switches.

Invensys recommends that the selected multimode fiber optic cabling have the following charac-teristics:

♦ 62.5 micron core/125 micron cladding

♦ Maximum allowable signal loss = 1 dB/km at a wavelength of 1300 nm

♦ Maximum allowable signal loss = 3.5 dB/km at a wavelength of 850 nm.

Cables with different characteristics can be used, but maximum transmission distance may be reduced. Cable requirements, such as flexibility, rodent protection, fire retardancy, and durability, depend on the particular application. Check with your cable installer/vendor for a list of applica-tion-specific cable characteristics.

Multimode fiber cable offered by Invensys that meet the recommended specifications are given in Table 2-2, Table 2-3, and Table 2-4. Copper cable offered by Invensys that meet the recom-mended specifications are given in Table 2-5.

Refer to the diagrams in the documentation included with your Invensys qualified Ethernet switches for detailed information on cabling.

Table 2-2. Multimode Fiber Cables with LC Connectors

Cable Length Connector

Type

Minimum Bend

Radius

P0972UN 0.5 m (1.5 ft) (gray) LC

2 inches

P0972VG 0.5 m (1.5 ft) (orange) LC

P0972UJ 1 m (3 ft) LC

P0972TN 3 m (9.8 ft) LC

P0972TP 15 m (50 ft) LC

P0972TQ 50 m (165 ft) LC

P0972WX (50 micron/mode conditioning cable)

3 m (10 ft) LC

Table 2-3. Multimode Fiber Cables with MT-RJ to ST Connectors

Cable LengthConnector

Type

Minimum Bend

Radius

P0972KV 3 m (9.8 ft) MT-RJ

2 inchesP0972KW 15 m (50 ft) MT-RJ

P0972KX 50 m (165 ft) MT-RJ

60


Single Mode Fiber Optic Cabling

Each switch must be equipped with an appropriate uplink module in order to use single mode fiber optic cable. Invensys recommends that the selected single mode fiber optic cabling have the following characteristics:

♦ 8-10 micron core/125 micron cladding

♦ Typical allowable signal loss = 0.5 dB/km at a wavelength of 1300 nm or 0.275 dB at a wavelength of 1550 nm.

Cables with different characteristics can be used, but maximum transmission distance might be reduced. Cable requirements, such as flexibility, rodent protection, fire retardancy, and durability, depend on the particular application. Check with your cable installer/vendor for a list of applica-tion-specific cable characteristics. Invensys does not offer single mode fiber optic cables. SMF cables are customer supplied.

Transmission distances for single mode fiber optic cable are given in Table 2-6.

Invensys does offer single mode fiber optic cables for use with patch panels. The MESH control network uses cables with MT-RJ connectors which have a pair of fibers which crossover within the cable (TX-to-RX). When patch panels are used on site, if they are constructed using MT-RJ connectors, consider the number of panels in series between end devices which need to be built. For instance, if a cable run is terminated with two patch panels and also connects to end devices, and the patch panel was constructed so that the cable is straight-through, then using two cross-over cables (one at each end) will not work.

Table 2-4. Multimode Fiber Cable with LC and SC Connectors

Cable Length Connector Type

P0972WW 3 m (9.8 ft) Two ceramic-type LC connectors on one end, with SC connectors on the other end

Table 2-5. Copper Cables with RJ-45 Connectors

Cable LengthConnector

Type

P0972UB 0.5 m (1.5 ft) RJ-45

P0971XK 3 m (9.8 ft) RJ-45

P0971XL 15 m (50 ft) RJ-45

P0972MR 30 m (110 ft) RJ-45

P0971XM 50 m (165 ft) RJ-45

P0971XN 100 m (330 ft) RJ-45

Table 2-6. Single Mode Fiber Optic Cable - Maximum Transmission

Application Transmission Protocol Max. Distance

Switch to Switch 1000Base-LX 10000 m (32808 ft)SMF cable is supplied by the customer

61


Connector Types for Switches

Refer to the documentation included with your Invensys qualified Ethernet switches for their connector types and uplink ports.

When patch panels are used on site, and if they are constructed using MT-RJ connectors, consider the number of panels in series between end devices which need to be built. For instance, if a cable run is terminated with two patch panels and also connects to end devices, and the patch panel was constructed so that the cable is straight-through, then using two crossover cables (one at each end) will not work.

Patch Panels

The MT-RJ jumper cables go straight through and the connector on the switch serves as the crossover. When patch panels are created using MT-RJ connectors, if the fiber run between two patch panels has a crossover, then the jumpers supplied by Invensys will not work. See Figure 2-20.

Table 2-7. Single Mode Fiber Optic Jumper Cables

Description Length Part Number

Single Mode Fiber (SMF) fiber optic patch cable w/ LC and SC connectors. Cable to be configured as a null modem (i.e., input/output crossed)

3 m (9.8 ft) P0973EW

Single Mode Fiber (SMF) fiber optic patch cable w/ LC and ST connectors.

3 m (9.8 ft) P0973EX


15 m (50 ft) P0973FY


50 m (165 ft) P0973FZ

Single Mode Fiber (SMF) fiber optic patch cable w/ LC - LC connectors.

3 m (9.8 ft) P0973FV


15 m (50 ft) P0973FW


50 m (165 ft) P0973FX

62


Figure 2-20. Switch to Switch and Switch to Patch Panel Connections

FCP270, ZCP270, FCM100Et and FCM100E Fiber Signal CablingThe cables listed in Table 2-8 are offered for use in making the 100 Mb MESH control network fiber optic cable connections. Whether these items are shipped to your location depends on the 100 Mb network cabling configuration specified at the time of system purchase.

Fiber optic cabling connects to the Ethernet 100 Mb switches using one MT-RJ connector and connects to the splitter/combiner for the FCP270, ZCP270, or FCM100Et using two ceramic type LC fiber optic connectors. These same connectors can connect the switches directly to a FCM100E. The maximum optical insertion loss though each connector must be equal to or less than 0.5 db. For the fiber optic cable lengths, refer to Table 2-8. Figure 2-21 shows the cabling for the FCP270, ZCP270, FCM100Et, FCM100E or patch panels.

Switch to Switch Connection

Switch to Patch Panel Connection

Switch

TXRX

TXRX

TXRX

TXRX

P1

P4

Switch

RXTX

RXTX

RXTX

RXTX

P4

P1

Switch

TXRX

TXRX

TXRX

TXRX

P1

P4

Switch

RXTX

RXTX

RXTX

RXTX

P4

P1

Patch

TXRX

TXRX

TXRX

RXTX

RXTX

RXTX

Patch

TXRX

TXRX

TXRX

RXTX

RXTX

RXTX

TXRX

RXTX

TXRX

RXTX

63


Table 2-8. Fiber Optic Cables

Part Number Connector Length Color Use

Cable Material

P0972TR LC toMT-RJ

3 m (9.8 ft) No specified color

Switch to Split-ter/Combiner FCP270/ ZCP270/ FCM100Etor directly to FCM100E (without Split-ter/Combiner)

MMF 62.5/125 micron

P0972TS 15 m (49.5 ft)

P0972TT 50 m (165 ft)

- Customer sup-plied, over 50 m (165 ft) to 2 km (6600 ft) fiber optic cable

P0972 UJ LC to LC 1.0 m (3 ft)

P0972TN 3 m (10 ft)

P0972TP 15 m (50 ft)

P0972TQ 50 m (165 ft)

P0972ZQ 0.25 m (1.0 ft)

P0972UN 0.5 m (2.1 ft) Gray color for A Bus

P0972VG 0.5 m (2.1 ft) Orange color for B Bus

P0972VD ST toMT-RJ

3 m (9.8 ft) No specified color

InterfaceMT-RJ devices to an ST-type patch panel

MMF 62.5/125 micron

P0972VE 15 m (49.5 ft)

P0972VF 50 m (165 ft)

P0972WW LC to SC 3 m (9.8 ft) Interface legacy P0972LQ (GPIM-01) to P0972WT (MGBIC-LC01)

Mode condi-tion MMF

P0972WX 3 m (9.8 ft)

P0972QP SC to SC 3 m (9.8 ft) SMF

P0973EW LC to SC 3 m (9.8 ft) Interface legacy P0972LR (GPIM-09) to P0972WU (MGBIC-LC09)

SMF 10 micron

P0973EX LC to ST 3 m (9.8 ft) Interface P0972WU (MGBIC-LC09) to an ST patch panel

64


Figure 2-21. FCP270, ZCP270, FCM100Et and FCM100E Signal Cabling

F

iber

100

Bas

e-F

X S

WIT

CH

W/M

TR

J C

ON

NE

CTO

RS

Cop

per

100B

ase-

TX

SW

ITC

H

W/R

J-45

CO

NN

EC

TOR

S

Cop

per

100B

ase-

TX

SW

ITC

H

W/R

J-45

CO

NN

EC

TOR

S

F

IBE

R O

PT

ICC

AB

LE G

RO

UP

E

CO

PP

ER

CA

BLE

GR

OU

P A

CA

BLE

GR

OU

P A

100B

ase-

TX

, CAT

5P

0972

UB

= 1

mP

0971

XK

= 3

mP

0971

XL

= 1

5 m

P09

72M

R =

30

mP

0971

XM

= 5

0 m

P09

71X

N =

100

m

NO

TE

: Pro

cess

or I/

O c

ards

are

illu

stra

ted

for

cabl

ing

purp

oses

onl

y.

OP

TIO

NA

L 1

GB

it U

PLI

NK

S.

RE

QU

IRE

UP

LIN

K G

BIC

MO

DU

LES

(P09

72W

T, P

0972

WU

, P09

72Y

L,P

0972

YQ

, P09

73G

H, P

0973

GJ,

P09

73F

T, o

r P

0973

FU

)

F

iber

100

Bas

e-F

X S

WIT

CH

W/M

TR

J C

ON

NE

CTO

RS

* =

NU

LL M

OD

EM

CA

BLE

# =

1:1

CA

BLE

CO

PP

ER

(ie

, Cu)

ET

HE

RN

ET

CA

BLE

FIB

ER

(ie

, FO

) E

TH

ER

NE

T C

AB

LE

GR

EE

N

MA

GE

NTA

ZC

P27

0Z

CP

270

CO

NN

EC

TIO

N T

O Z

CP

270

FCP270 (P0917YZ)

CO

NN

EC

TIO

N T

O F

CP

270

OPTICALSPLITTER

OPTICALSPLITTER

P09

72U

N (G

RAY

)

P09

72V

G (O

RA

NG

E)

P09

26A

HFCP270 (P0917YZ)

FCP270 (P0917YZ)

OPTICALSPLITTER

OPTICALSPLITTER

P09

72U

N (G

RAY

)

P09

72V

G (O

RA

NG

E)

P09

26A

H

P09

22K

Y10

/100

Bas

e-T

XE

TH

ER

NE

T

PC

I CA

RD

P09

22K

Y10

/100

Bas

e-T

XE

TH

ER

NE

T

PC

I CA

RD

FIB

ER

CA

BLE

GR

OU

P E

MM

F w

ith M

T-R

J/LC

CA

BLE

SP

0972

TR

= 3

mP

0972

TS

= 1

5 m

P09

72T

T =

50

mC

US

TOM

ER

SU

PP

LIE

D =

>50

m, <

2 k

m

TWO SINGLEETHERNET CARD(S)

OPTICALSPLITTER

P09

26A

H

CO

NN

EC

TIO

N T

O F

CM

100E

t

or

FC

M10

0E

OPTICALSPLITTER

FB

MF

BM

FB

M

FB

M

P09

72U

N (G

RAY

)

P09

72V

G (O

RA

NG

E)

DIN RAILMOUNTED

TH

E M

ES

HC

ON

TR

OL

NE

TW

OR

K

I/A S

erie

s

I/A S

erie

sI/A

Ser

ies

I/A S

erie

sI/A

Ser

ies

I/A S

erie

s

I/A S

erie

s

SIN

GLE

FC

M

A

FCM100E** (P0972ZA)

I/A S

erie

s

B

FCM100E** (P0972ZA)

A B

FCM100Et** (P0926GS)

RE

DU

ND

AN

T P

AIR

SIN

GLE

FC

P

A B AA B

B

FAU

LT T

OLE

RA

NT

PA

IR

SW

ITC

H G

RO

UP

AS

WIT

CH

GR

OU

P A

SW

ITC

H G

RO

UP

B

SW

ITC

H G

RO

UP

B

A B

FCM100Et** (P0926GS)

OPTICALSPLITTER

OPTICALSPLITTER

I/A S

erie

s

FB

M

I/A S

erie

s

FB

M

PR

OC

ES

SO

R C

AR

DS

(Wor

ksta

tion)

P09

72Z

Q

P09

72Z

Q

P09

72Z

Q

** A

lso,

FC

M10

0E m

odul

es (

P09

72Z

A)

may

be

used

in p

lace

of F

CM

100E

ts.

The

FC

M10

0E m

odul

es d

o no

t req

uire

split

ter/

com

bine

rs fo

r co

nnec

tion

to th

e sw

itche

s.

65


Twisted-Pair CablingCategory 5 (CAT5), shielded twisted-pair (STP) copper cable is used in The MESH control net-work for connecting switches to RJ-45 ports on other devices. CAT5 cable can be used, with the appropriate uplink module, to connect copper switch ports to other copper switch ports or media converters.

Transmission distances for CAT5 copper cable are given in Table 2-9.

Table 2-10 lists the prefabricated CAT5 STP cables with RJ-45 connectors that are offered by Invensys.

NOTEPlenum grade jackets are supplied on cables over three meters in length that are offered by Invensys. This applies to both fiber optic and CAT5 copper cables.

Null HubA null hub is a very short cable that has a male RJ-45 connector on one end and a female RJ-45 connector on the other. The transmit and receive wires are reversed, so when it is connected to the end of a straight-through cable, the resulting cable system acts as a crossover cable. The null hub is used to connect switches that do not have an MDI crossover port or an auto MDI/MDI-X port.

Table 2-9. CAT5 Cable - Maximum Transmission Distance

Application Transmission Protocol Max. Distance

Media converter to RJ-45 port, switch to switch, port to port

100Base-TX and 1000Base-T

100 m (328 ft) for all applications

Table 2-10. Prefabricated CAT5 STP Cables with RJ-45 Connectors

CAT5 Cable LengthConnector

Type

P0972UB 1 m (3.3 ft) RJ-45

P0971XK 3 m (10 ft) RJ-45

P0971XL 15 m (50 ft) RJ-45

P0972MR 30 m (100 ft) RJ-45

P0971XM 50 m (165 ft) RJ-45

P0971XN 100 m (330 ft) RJ-45

Table 2-11. Null Hub

Null Hub Part Number Length

P0971PK 0.3 m (1 ft)

66

3. Installation (Cabling)
This chapter describes the steps necessary to install the cabling that enables The MESH control network equipment to communicate.

Installation of The MESH control network consists of locating the individual components and then making connections between them using fiber optic or copper cable. The following para-graphs provide the information necessary for the cabling installation of the high performance sys-tem network.

NOTERefer to the documentation included with your Invensys qualified Ethernet switches to install these switches and the media converters as part of The MESH control network.

Connecting The MESH Control Network Components

Fiber Optic Cabling GuidelinesThe following guidelines should be followed when making fiber optic connections between devices in The MESH control network:

♦ Minimum bend radius - Fiber optic cable can be damaged if it is bent too sharply. Maintain a minimum bend radius of two inches when using fiber optic cable supplied by Invensys. Follow the cable manufacturers guidelines if third-party fiber optic cable is used.

♦ Cable straps - Cable straps, if used, should be designed for use with fiber optic cable. Hard plastic cable ties can damage fiber optic cable and are not recommended.

♦ Dust caps - Dust caps should remain in place to protect the polished cable ends over the cable ends and in the switches while cables are being routed. Dust caps should be installed on unused switch ports and device ports.

Interconnecting Ethernet SwitchesDepending on which switch model has been selected as the root network switch, additional switches may be connected in one of several ways. They are summarized in Table 3-1 below. Refer to the “Ethernet Switch Interconnection Diagrams” in the documentation included with your Invensys qualified Ethernet switches for detailed cabling diagrams.

67

B0700AZ – Rev L 3. Installation (Cabling)

CAUTION!If Fast Ethernet “Device” ports are used as uplink ports between switches on an N-Series chassis switch configuration, it is very important that the switch spanning tree protocol settings be configured correctly. Failure to do so will cause system deg-radation during switch failover, causing excessive packet flooding possibly resulting in device failures. (Refer to The MESH Control Network Operation, and Switch Instal-lation and Configuration Guide (B0700CA).)

CAUTION!Do not make direct interconnection between ports of the same switch (i.e. loop-back). Loopbacks can result in network failures upon a loss or failure of RSTP.

Also, refer to page 61 for a list of single mode fiber optic cables for use at patch panels.

NOTEAll graphics of switches and media converters below are intended as generic illustra-tions of networking concepts and do not necessarily reflect the currently offered products.

NOTEThese sections pertain to both standard switches and chassis switches.

Uplink Port to Uplink PortThe Ethernet switches can be connected to each other through the use of an optional uplink port. Depending upon which uplink module is installed, the switch is able to communicate over single mode or multimode fiber optic cable, or CAT5 copper cable. Transmission via uplink ports is made using the Gigabit Ethernet protocol. The fiber optic cable uses LC type connectors and should be designed for the mode (single mode vs. multimode) of uplink module that is installed in the Ethernet switch’s interface slot. Figure 3-1 shows a connection between two Ethernet switches using their fiber optic uplink ports.

Table 3-1. Methods of Connecting Ethernet Switches

Root Switch Port End Switch Port

Uplink port (Gigabit Ethernet) copper and fiber

Uplink port (Gigabit Ethernet)

Fiber optic port (100Base-FX) Fiber optic port (100Base-FX)

Fiber optic port (100Base-FX) RJ-45 (100Base-TX)

RJ-45 (10Base-T/100Base-TX) RJ-45 (10Base-T/100Base-TX)

RJ-45 (10Base-T/100Base-TX) Fiber optic port (100Base-FX)

68

3. Installation (Cabling) B0700AZ – Rev L

Figure 3-1. Switch-to-Switch Fiber via Uplink Port

Managed switches can be connected directly to other switches connecting each switch’s fiber optic ports with fiber optic cable.

NOTEMake sure that the transmit (TX) port of one switch is connected to the receive (RX) port of the other switch. The standard Invensys LC to LC cable ensures proper TX to RX connections.

Fiber optic cable can be used to connect fiber switches to other fiber switches using each device’s 100Base-FX fiber ports. Multimode fiber cable with MT-RJ connectors is connected between a port on each switch. Figure 3-2 gives an example of a port-to-port connection via fiber optic cable. However, even though this method can be used for uplink ports it is not recommended due to the 100 Mb port's bandwidth capabilities. This method of uplinks must not be used for the Security Enhanced Configuration topology designs. Normally 1 Gb uplink connections should be used to interconnect switches in all standard or enhanced network designs.

Figure 3-2. Port-to-Port Connection via Fiber Optic Ports

Fiber Switch

62.5/125 micron Single/Multimode

w/LC-type ConnectorsFiber Optic Cable

1000Base-SX, 1000Base-LX or 1000Base-ZX

Fiber Switch

Fiber SwitchFiber Switch

62.5/125 micron Multimode

w/LC ConnectorsFiber Optic Cable

100Base-FX

69

B0700AZ – Rev L 3. Installation (Cabling)

CAUTION!If Fast Ethernet “Device” ports are used as uplink ports between switches, it is very important that the switch spanning tree edge-port protocol settings be configured correctly. Failure to do so will cause system degradation during switch failover, caus-ing excessive packet flooding possibly resulting in device failures. (Refer to The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA).)

RJ-45 Port to RJ-45 PortSwitches can be connected to other switches through an RJ-45 port on each switch. It is necessary that the first switch’s transmit signal is connected to the second switch’s receive line. The same is true for the second switch’s transmit and receive signals. However, even though this method can be used for uplink ports it is not recommended due to the 100 Mb port's bandwidth capabilities and the RJ-45 auto-negotiation characteristics. This method of uplinks is not recommended for the enhanced network topology designs. Normally 1 Gb uplink connections should be used to interconnect switches in all standard or enhanced network designs. Figure 3-3 shows a connection between a copper switches with auto MDI/MDI-X ports.

Figure 3-3. Connecting Switches via RJ-45 Ports

In the event that neither switch has an MDI/MDI-X port, a separate crossover cable (P0971PK - with a 1 ft cable), called a null hub, must be used between the switches.

Switch ConfigurationRefer to The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA) and the documentation included with your Invensys qualified Ethernet switches for instructions on configuring them for The MESH control network.

Copper Switch

100Base-TX

AutoMDI/MDI-X

AutoMDI/MDI-X

Copper Switch

70

4. Maintenance
This chapter provides information on identifying the cause of problems with The MESH control network.

The MESH Control Network AddressesAll I/A Series equipment IP addresses for The MESH control network are 151.128.81.x (where x is a number between 1 and 254 that identifies the specific Ethernet switch). IP addresses assigned to the primary port of controllers (FCP270, ZCP270), FCM100Ets, FCM100Es, and worksta-tions are 151.128.y.z (where y is a number between 152 and 191, and z is a number between 1 and 254). The IP addresses for the alternate ports of FCM100Ets, FCM100Es or workstations are 151.128.w.z (where w is derived from y above as w = y - 128, and z is the same value as above). The IP addresses for the primary I/O controller ports of ZCP270 are 151.128.v.z (where v is derived from y above as v = y + 64, and z is the same value as above). The IP addresses for the alternate Input/Output Controller (IOC) ports of ZCP270 are 151.128.u.z (where u is derived from y above as u = y - 64, and z is the same value as above). See Table 4-1.

Fault tolerant FCP270s and ZCP270s use the same IP addresses for both modules of the fault tol-erant pair. Additionally, the ZCP270 requires a third and fourth IP address for its Input/Output Controller (IOC) ports. These also are derived and assigned according to Table 4-1.

IP addresses for all devices on The MESH control network (workstations, controllers, managed switches, FCM100Ets and FCM100Es) are assigned by the system configurator (SysDef or IACC). For information on using the system configurator, refer to:

♦ I/A Series System Definition: A Step-by-Step Procedure (B0193WQ) or

♦ I/A Series Configuration Component (IACC) User’s Guide (B0400BP).

Table 4-1. IP Address Assignments

I/A Series Release Port IP Address

v8.0 or later Network Monitoring Devices 151.128.82.1 through 151.128.82.254

v8.0 or later Primary Port 151.128.152.1 through 151.128.191.254

v8.0 to v8.1.x Alternate Port1

1. At v8.2, the intermediate driver was changed so that the Alternate Port IP is never used.

151.128.24.1 through 151.128.63.254

v8.0 or later Primary IOC Port 151.128.216.1 through 151.128.255.254

v8.0 or later Alternate IOC Port 151.128.88.1 through 151.128.127.254

v8.0 or later Switches 151.128.81.1 through 151.128.81.254

Pre-v8.0 Single Node 151.128.8.65 through 151.128.8.126

Pre-v8.0 Multi-Node 151.128.16.65 through 151.128.23.254

Pre-v8.0 Switches 151.128.79.1 through 151.128.79.254 or 151.128.80.1 through 151.128.80.254

71

B0700AZ – Rev L 4. Maintenance

Once the configurator has assigned the primary IP address, each Ethernet switch must be config-ured with its assigned IP address using the Command Line Interface (CLI) Entries or I/A Series Switch Configurator Application Software (discussed in the switch configuration section of the documentation included with your Invensys qualified Ethernet switches). One IP address is assigned to each non-Chassis Ethernet switch in The MESH control network. For the N1 Chassis switch (P0973AR), the chassis is assigned one IP address. For the N3 Chassis switch (P0973AS), the chassis is assigned one IP address for each of its three blades. For the N7 Chassis switch (P0972YE), the chassis is assigned one IP address for each of its seven blades.

The system configurator (SysDef or IACC) assigns a MAC (Media Access Control) address for each FCP270, ZCP270, FCM100Et, and FCM100E on the network. The primary port MAC addresses are assigned from the range, 00006CC00000 to 00006CC03FFF. The alternate port MAC addresses are derived from the primary port MAC addresses by “ORing” the primary net-work MAC address with 000000004000, yielding a range of 00006CC04000 to 00006CC07FFF. The MAC addresses for the primary I/O controller ports of ZCP270 are derived from the primary port MAC addresses by “ORing” the primary network MAC address with 000000008000, yielding a range of 00006CC08000 to 00006CC0BFFF. The MAC addresses for the alternate I/O controller ports of ZCP270 are derived from the primary port MAC addresses by “ORing” the primary network MAC address with 00000000C000, yielding a range of 00006CC0C000 to 00006CC0FFFF.

Workstations, FCP270, ZCP270, FCM100Et, FCM100E and FBMs are assigned letterbugs to identify the module by the system configurator. FBMs are not assigned IP or MAC addresses. Let-terbugs and IP addresses can be changed using the system configurator

General Troubleshooting GuidelinesWhen a problem occurs with The MESH control network, it is usually best to take an organized approach to diagnosing the cause. A random or scattered approach generally takes longer and can make it more difficult to track down the cause of the problem by introducing additional unknowns. The following sections will help to determine the cause of a network problem.

Characterize the ProblemThe initial step in diagnosing network problems is to understand what kind of problem exists.

♦ Has the problem always existed or has it just started?

New problems are sometimes caused by a change in The MESH control network con-figuration. Check to see whether there have been recent changes to The MESH con-trol hardware, software, or configuration files.

♦ Is the problem constant or intermittent?

Constant problems are often the result of a component failure or a change to The MESH control network configuration. Intermittent problems can be caused by envi-ronmental factors such as excess heat, electrical noise, poor contacts, or high attenua-tion.

♦ What kind of error indication is occurring?

Device communication failure - inability to communicate with one or more devices on The MESH control network. This happens when the information path has been broken or interrupted. Possible causes are damaged or misconnected cabling, a failure in a network device, or high attenuation.

72

4. Maintenance B0700AZ – Rev L

Data Loss - incorrect data is arriving at one or more devices. The data path is intact, but the data is being corrupted along the way. This can be caused by failing network devices, environmental factors, or exceeding transmission distance limits.

Error message - software has detected a problem. This can be caused by any number of things. The text of the error message indicates what kind of problem has been detected.

Determine Which Devices are AffectedIdentifying the devices that are affected helps to determine where the cause of the problem lies. For example, if all the unreachable devices are connected to the same Ethernet switch, then you should probably start troubleshooting at that switch.

When troubleshooting a Security Enhanced Configuration with LDP deployed, it is highly rec-ommended that you refer to the section “Loop Detection Monitoring and Maintenance” in The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA) before continuing.

Follow the steps below to identify which device or devices are affected.

1. Are the problems limited to specific devices (stations, switches), or are they general in nature?

2. Are the problems constant or are they intermittent?

3. Can the devices be reached using the ping command?

4. Does the ping -s command report that packets are being dropped?

5. What does System Management Displays indicate? Refer to “System Management Displays” on page 74.

6. Use a network topology map to determine the physical location of the affected devices.

Troubleshoot the Affected Devices1. Check activity and status indicators on the affected devices.

2. Verify that the devices are getting power (check Status LEDs).

3. Check that the devices are correctly cabled and that the cables are not damaged (swap a suspect cable with a known good cable).

4. Verify that the limit on distance between devices has not been exceeded. Distance is dependent on transmission mode and cable type.

5. If fiber optic cable is being used, verify that the optical budget (total signal loss - dB), bandwidth, and dispersion are within specification.

6. Verify that traffic rates to the affected devices are not excessive.

7. Check System Management error counters. Refer to System Management Displays (B0193JC).

8. Verify that the devices are properly configured. Also check that the adjacent device in the network is correctly configured as well.

73


NOTEBefore removing a switch from the network, be sure to record its existing configura-tion. This information is necessary when the replacement switch is configured.

NOTEMake sure a replacement switch is correctly configured before adding it to the net-work. Any uplink ports should be installed before the switch is configured. Refer to the documentation included with your Invensys qualified Ethernet switches for more information.

System Management DisplaysThe I/A Series System Management Display Handler (SMDH) obtains current and historical information about the system, and displays this information in System Management displays. SMDH uses SNMP to access the switches to obtain status information from each switch. With regard to The MESH control network and associated Ethernet switches, SMDH provides the fol-lowing displays:

♦ A Switched Network View of all the configured switches

♦ A Switch Domain display that shows all The MESH control network stations in the selected Switch’s domain.

♦ A Switch Ports Display which shows each port of the switch that was selected on the Domains display

♦ Detailed equipment information (EQUIP INFO) and equipment change (EQUIP CHG) displays for each port of the selected Ethernet switch.

Accessing SMDH Switch Network DisplaysSystem Management Displays are accessed from the FoxView™ application.

To access System Management on Windows® based systems or Solaris® based systems with the FoxView application, refer to System Management Displays (B0193JC).

The initial System Management Display, System Monitor Domains, appears. From this display you can select a system monitor and navigate through the Domain Display(s).

All switches are shown in the system monitor domain as stations designated by their letterbug. To access displays for The MESH control network switches, select (click-on) the menu bar soft keys indicated in Figure 4-1.

74


Figure 4-1. Accessing SMDH Switched Network Displays

Domains

SwitchedNetwork Display

Click Switch

Equipment Change Display

System Monitor

Click SW NTWRKDomains

Click EQUIP CHGClick EQUIP INFO

Click CONFIG INFOClick NEXT LEVEL

SwitchDomain Display

Click Switch


Click CONFIG INFOClick NEXT LEVEL

Equipment Information Display

Equipment Configuration Display

SwitchPorts Display

Click Port


Equipment Change Display

Equipment Information Display

(Figure 4-2)

(Figure 4-6)

(Figure 4-7)

(Figure 4-3)

(Figure 4-4)

(Figure 4-5)

(Figure 4-8)

(Figure 4-9)

75


Switched Network DisplayA typical Switched Network display is shown in Figure 4-2. The display is accessed from the Sys-tem Monitor Domains display by clicking on the SW NTWRK soft key.

It displays the letterbug “system name” of every switch in the network. Note that each card (blade) in an E7 Chassis switch is assigned a letterbug, where the N7 Chassis switch is only assigned one letterbug. Media converters and extenders are not assigned letterbugs.

From the Switched Network Display you can select a switch (click on the letterbug of a particular switch of interest) and then click on the EQUIP CHG, EQUIP INFO, CONFIG INFO, or NEXT LEVEL. Clicking NEXT LEVEL brings up the Switch Domain Display.

Figure 4-2. SMDH Switched Network Display

76


Switch Equipment Change DisplayA typical Switch Equipment Change display is shown in Figure 4-3. The display is accessed from the Switched Network display or the Switch Domain display.

The Switch Equipment Change display enables/disables station (switch) alarms and reports from being propagated to the I/A Series system. Table 4-2 describes only the available text fields in the order that they appear on the display pages, from left column to right column.

Figure 4-3. Switch Equipment Change Display

Table 4-2. Switch Equipment Change Display Actions

Action Description

ENABLE STATIONALARMING

Enables device status to affect the overall system status.

INHIBIT STATIONALARMING

Inhibits device status to affect the overall system status.

ENABLEALL REPORTS

Enables reports between the device and the I/A Series System Man-agement.

DISABLEALL REPORTS

Disables reports between the device and the I/A Series System Man-agement.

77


Switch Equipment Information DisplayA typical Switch Equipment Information display is shown in Figure 4-4. The display is accessed from the Switched Network display or the Switch Domain display. It shows the name, type, and status of the switch.

Table 4-3 describes only the available text fields in the order that they appear on the display pages, from left column to right column.

Figure 4-4. Switch Equipment Information Display

Table 4-3. Switch Equipment Information Display Fields

Field Description

STATION NAME Name (letterbug) of the selected station.RUN MODE On-line or Off-line (default) is displayed.FAIL ACK STATE Acknowledged (default) or Not Acknowledged is displayed. If the FAIL

STATE value changes from Not Failed to Failed, the FAIL ACK STATE value changes to Not Acknowledged to indicate this transition, and remains until you acknowledge the switch failure.

FAIL DEV ACK Acknowledged (default) or Not Acknowledged is displayed. If any of the switch ports fail and become unacknowledged, Not Acknowledged is displayed.

MAC ADDRESS The MAC Address of the station.TYPE The number of ports for a particular switch: Sw w/16 ports, Sw w/24

ports, Sw w/48 ports. For the N1, N3, N7 Chassis switches, displays “Ethernet Switch.” The ports for all switches are displayed on the Switch Ports Display.

78


FAIL STATE Fail or Not Failed is displayed.ALARMING STATE Alarming State indicates whether alarming is enabled or inhibited for

the switch. When alarming is inhibited, the System Monitor continues to indicate overall system and network health (a green SYS BAR).

FAIL DEV ATT Yes is displayed if one or more switch ports has failed; otherwise, No is displayed.

MT REPORT STATE Master Timekeeper Report State. GPS Not Configured is always dis-played for the switch equipment information display.

Table 4-3. Switch Equipment Information Display Fields (Continued)

Field Description

79


Switch Configuration Information DisplayA typical Switch Configuration Information display is shown in Figure 4-5. The display is accessed from the Domains display.

Table 4-4 describes the available text fields in the order that they appear on the display pages, from left column to right column.

Figure 4-5. Switch Configuration Information Display

Table 4-4. Switch Configuration Information Display Fields

Field Description

STATION NAME Name (letterbug) of selected station.SMON NAME System Monitor name for the selected station.IP ADDRESS IP Address for the selected station.SW CONNECTION Lists all of the switches to which this switch’s ports are connected such

as; SW001B, SW003A, SW004B.TYPE The number of ports for a particular switch: Sw w/16 ports, Sw w/24

ports, Sw w/48 ports, up to a Sw w/672 ports.SMON AP NAME The name (letterbug) of workstation that is hosting SMON.BOOT HOST Name (letterbug) of boot host for the switch.

80


Switch Domain DisplayThe Switch Domain display for a switch is shown in Figure 4-6. The display is similar for all switches.

From the Switch Domain Display you can select a switch (click on the letterbug of the switch) and then click on the EQUIP CHG, EQUIP INFO, CONFIG INFO, or NEXT LEVEL. The EQUIP CHG, EQUIP INFO, CONFIG INFO displays are the same as those that can be selected from the Switched Network Display. Clicking NEXT LEVEL brings up the Switch Switch Ports Display.

Figure 4-6. Switch Domain Display

The SWITCH DOMAIN DISPLAY indicates the status of The MESH control network Ethernet cables to/from the modules connected to the switch. If a fault in cable A or cable B is detected, the faulty Ethernet cable is identified by the mnemonic RA (Receive A), RB (Receive B), or RAB (Receive A and Receive B) appearing next to the receive cable as shown below:

RA RB

81


Switch Ports DisplayA typical Switch Ports Display for a 16-port switch is shown in Figure 4-7. The Switch Ports Dis-play, displays a maximum of 30-ports on one screen. Paging is enabled so that more than 30-ports can be shown on the display.

The carrot < next to the port number in Figure 4-7 indicates that the port alarming has been inhibited (see “Switch Port Equipment Change Display” on page 83). All unused ports must be inhibited, disabling switch alarming for that port. This stops false alarms from propagating upward in the I/A Series System Management.

The asterisk “*” adjacent to the port number in Figure 4-7 indicates that the port has failed “Lost Link” and should be investigated to determine the cause of the port failure. Refer to “General Troubleshooting Guidelines” on page 72 to aid in the isolation of the failure.

Figure 4-7. Switch Ports Display - Typical

82


Switch Port Equipment Change DisplayThe Equipment Change display for a switch is shown in Figure 4-8. The display is similar for all switch ports. Table 4-5 describes the available text fields in the order that they appear on the dis-play pages, from left column to right column.

Figure 4-8. Switch Equipment Change Display

Table 4-5. Switch Equipment Change Actions

Action Description

ENABLE DEVICE ALARMING Enables switch alarms to propagate upward in the I/A Series System Management.

INHIBIT DEVICE ALARMING Inhibits switch alarms from propagating upward in the I/A Series System Management. All non-used Ethernet ports must be inhibited for proper device monitoring.

83


Switch Port Equipment Information DisplayAn Equipment Information display for a port of an Ethernet switch is shown in Figure 4-9. The display is similar for all switches. Table 4-6 describes the available text fields in the order that they appear on the display pages, from left column to right column.

Figure 4-9. Switch Port Equipment Information Display - Typical

Table 4-6. Switch Port Equipment Information Display Fields

Field Description

NAME The port number of the switch displayed in SMDH as P01 through P378.

RUN MODE On-line or Off-line (default) is displayed. FAIL ACK STATE Acknowledged (default) or Not Acknowledged is displayed. If the

DEVICE STATE value changes from Not Failed to Failed, the FAIL ACK STATE value changes to Not Acknowledged to indicate this transition, and remains until you acknowledge the port failure.

WARNING STATE Yes is displayed if the device has a non-fatal error condition; otherwise, No is displayed.

NUMBER The port number 1 through 64.CURRENT STATE Port running or port failed.TYPE The switch port or the uplink of the switch.DEVICE STATE Failed is displayed if one or more devices connected to the switch has

failed; otherwise, Not Failed is displayed.

84


Indicators

Ethernet SwitchesFor the location and meaning of the indicators on a particular Ethernet switch, refer to the docu-mentation provided with the switch.

Media ConverterFor the location and meaning of the indicators on a particular media converter, refer to the docu-mentation provided with the device.

Fiber Optic Cable Handling and CleaningFiber optics communication relies on a clear path for its signals. Make every effort to install splices, connectors, and terminations as clean as possible, thus reducing their effects on optical data transmission.

Handling Fiber Optic Cable Consult the cable specifications for the cable you are installing. Mishandling the cable may cause damage that can alter its transmission characteristics requiring replacement of the cable. When handling fiber optic cabling:

ALARMING STATE Alarming State indicates whether alarming is enabled or inhibited for the device. When alarming is inhibited, the System Monitor continues to indicate overall system and network health (a green SYS BAR) while equipment is Failed or Off-line.

COMM STATE OK or Fail.ADMIN STATE Port Enabled or Port Disabled.INTERFACE STATE Interface state indicates the communications state of the port. Ethernet-

csmacd or software Loopback are normally displayed for The MESH control network. Other states that can be displayed are: other, regular1822, hdh1822, ddn-25, rfc877-x25, iso88023-csmacd, iso88024-tokenbus, iso88025-tokenRing, iso88026-man, starLan, pro-teon-10 Mb, proteon-80Mb, hyperchannel, fddi, lapb, sdlc, dsl, el, basic ISDN, proppointToPointSerial, ppp, eon, ethernet-3Mb, nsip, slip, ultra, ds3, sip, or fram-relay. These other states can be shown depending on the type of switch and communications employed.

Table 4-6. Switch Port Equipment Information Display Fields (Continued)

Field Description

85


WARNING!Never look directly into the end of a fiber cable or bulkhead adapter. Eye damage may result. Laser light can damage your eyes. Laser light is invisible. Viewing it directly does not cause pain. The iris of the eye will not close involuntarily as when viewing a bright light. Consequently, serious damage to the retina of the eye is pos-sible. Never look into the end of a fiber or at a connector, or a connector in an inspection microscope which may have a laser coupled to it. Should accidental eye exposure to laser light be suspected, arrange for an eye examination immediately.

♦ Turn off power to the equipment before cleaning or looking at fiber cable ends.

♦ Always use dust covers on the end of the fiber cable connector (see Figure 4-10, Figure 4-11 and Figure 4-12) or any bulkhead (including unused bulkheads)

♦ Do not pull or kink the cable as the glass strand in the middle might become damaged or broken

♦ Do not walk-on, step-on, or crush the cable as the glass strand in the middle might become damaged or broken

♦ Keep bend radii no less than two inches

♦ Use specialized optical cable raceways and plenums whenever available

♦ Never use tie wraps as you would with electrical cables

♦ When using optical connectors, insert or remove the ferrule straight into the sleeve. Minimize wiggling the connection as this may loosen the tight fit

♦ For SC connectors, orient the prominent key on the connector body with the slot in the bulkhead adapter. Push the connector until it clicks. To remove, pinch the con-nector body between your thumb and finger, and gently pull straight out (see Figure 4-10)

Figure 4-10. SC Connector, Typical

♦ For MT-RJ connectors, orient the connector body with the slot in the bulkhead adapter. Push the connector until it clicks. To remove, push the release tab between your thumb and finger, and gently pull straight out (see Figure 4-11)

Release TabsDust Covers

86


Figure 4-11. Multimode MT-RJ Connector

♦ For LC connectors, orient the connector body with the slot in the bulkhead adapter. Push the connector until it clicks. To remove, push the release tab between your thumb and finger, and gently pull straight out (see Figure 4-12).

Figure 4-12. Multimode Duplex LC Connector

Cleaning Fiber Optic Cable

Contamination of Fiber Optic Connectors and SocketsFiber optic cable connector parts are particularly sensitive to contamination which reduces the light transfer from one side of the interface to the other. Finger oils, dust, fuzz, and so forth, can attenuate the photon transfer across the cable connector to socket interface. Experience has shown that minute quantities of contaminants can have major effects on the communication between controllers and workstations.

The first line of defense is to prevent contamination from building up on the sensitive fiber optic surfaces. The second approach is to remove the contamination once it has occurred. Prevention of contamination is easier and cheaper than cleaning up after the fact.

Release Tab Dust Cover

Release TabDust Covers

87


Contamination PreventionThe easiest way to prevent contamination from affecting fiber optic sockets is to keep the protec-tive covers (see Figure 4-10, Figure 4-11 and Figure 4-12) on all sockets which are not currently in use. You should retain these dust covers (rubber plugs) in a place where they can be retrieved or maintain a ready supply of covers so that they may be inserted into sockets when cables are removed. All fiber optic switches, control modules (for example, FCP270 and ZCP270), and interface modules (for example FCM100Et, FCM100E, Fiber NIC, ATS) come with protective plugs installed in each socket. Figure 4-13 shows an example of microscopic lint which has lodged in a fiber optic LC socket. The contaminant in this example totally prevented the port from func-tioning. This type of contamination similarly affects MT-RJ and SC connectors, especially when static charges on the connectors attract charged particles.

Figure 4-13. Lint in Fiber Optic LC Socket

Cleanliness is equally necessary with FO cable connectors’ plugs. Finger oils as well as dust and lint can negatively affect the cable plug’s transmission abilities. As with the sockets, the easiest defense is to keep the dust covers, which come shipped on the connectors, in place until just before insertion into a socket.

Contamination RemovalIn spite of following the best contamination prevention procedures, you will eventually encounter a “dirty” connection, in which case you must clean the connection. There are basically three methods to remove contamination from FO connections; blowing, chemical washing, and abra-sion.

The blowing method consists of removing dust or lint from a connector with a blast of dusting gas such as Tetrafluoroethane, usually from an aerosol can. This is effective for removing material in depressions (such as in a socket) which is loosely coupled to the surface, for example, as by static electricity. Because dust can be transferred into connector sockets through the normal inser-tion of cable plugs, it is good practice to dust off the connector before plugging it in.

Chemical washing basically consists of flushing or wiping the face of a fiber with isopropyl alco-hol. The alcohol can be wiped on the fiber face by applying the fluid to a lint-free wipe or lint-free swab, or by using pre-saturated wipes. These alcohol saturated wipes are most useful in cleaning the connectors on the ends of cables where the exposed fibers are most easily accessible. The alco-hol saturated swabs are needed for the female sockets in which the fiber is located in deep recesses.

88


The abrasion technique is useful for cleaning cable connectors such as MT-RJ style; it is not used to clean sockets. It is a dry method which does not require isopropyl alcohol. It is generally avail-able in the form of a fabric tape housed in a dispenser which exposes a short length of tape over which you wipe the end of the connector. The fabric tape acts as a fine abrasive which removes contamination from the glass fiber end.

Examples of cleaning products are listed below. It is suggested that the user of fiber optic cables keep on hand similar items so you can quickly get back on-line when a contamination situation arises. It is also highly recommended that you observe the storage and use suggestions provided by the vendor of these cleaning products since some are classified as hazardous material. Obtain and use the Material Safety Data Sheets (MSDS) and U.S. Department of Transportation DOT-E 10232 literature associated with these products as handling guides.

Examples of Fiber Optic Connector Cleaning ProductsThe following products are available through various distributors:

♦ Techspray Fiber Optic Cleaning Kit, Part number 1602 (see: www.techspray.com) Contains: Isopropyl alcohol, alcohol saturated wipes, aerosol duster, swabs, and wipes.

♦ Techspray Aerosol Duster, Part number 1671-15S (see: www.techspray.com) Contains an Aerosol duster (can of compressed Tetrafluoroethane gas).

♦ Cletop S Cassette cleaner, P/N 14110611 (see: www.cletop.com)An abrasive-type cable connector cleaner.

♦ Cletop 1.25 mm Q-tip (swab) stick cleaner, P/N 14100401(see: www.cletop.com)

♦ Corning Cable Systems, P/N 2104359-01(see: www.corning.com/cablesystems)An abrasive-type cleaning cassette.

89


90

5. Combining Two or More MESH Control Networks
This chapter provides information on combining two or more instances of The MESH control network.

OverviewWhen combining two or more MESH networks, many variables must be taken into consider-ation. Combining MESH networks actually refers to combining I/A Series systems. When I/A Series systems are combined, a new committal must be generated which contains all the sta-tions from the combined systems. All workstations must be recommitted and any station that receives a new letterbug, NSAP, MAC address or IP address will have to be rebooted.

The following examples cover most of the basic situations and list the essential principles which must be followed. Care must be taken when combining networks in order to prevent system or network problems and even possible system shutdown.

Rules:

1. A complete inventory of all device IP, MAC and NSAP addresses on all networks must be performed to ensure no duplicate addresses are present.

2. Establish the topology of both networks obtaining the following required informa-tion.

a. Which switches are designated as the root bridge switches (Root/Back-up root) on all networks

b. What topology configuration are the networks (Star, Tree, Ring, etc.), and are they compatible with each other (with the consideration of the step c below)?

c. When the two networks become, as one, will any two switch communications exceed the maximum limit of seven hops?

d. If VLANs are deployed on one or more networks do the VLAN assignments match between the networks?

e. If Loop Detection Policy (LDP) is deployed on one or more networks, how will they function between each other when considering the rule requirements for LDP?

3. When combining two or more “live” networks, the process can be dangerous. It is very important that all network and device components attached to the network be 100% functional and the functioning status of all switches must be known (root, back-up root, uplink port status, configuration and location).

The following is the process by which two or more networks are combined.

Planning StageOne must assume if two or more networks are being combined that there is more than one I/A Series system in operation. To combine two or more I/A Series systems, the systems in whole

91

B0700AZ – Rev L 5. Combining Two or More MESH Control Networks

must be re-committed. When re-committing the systems, the Bridge switch functionality must be considered - see below.

Station Addressing1. If duplicate IP, MAC and NSAP addresses on I/A Series devices or stations exist

within the networks, then those stations or devices will have to have new addresses assigned.

2. Stations and devices with new addressing will have to be marked PCHANG and rebooted after the whole system has been committed.

3. If duplicate IP addresses exist on a network switch, the switch must be re-configured with a new IP address and the switch host workstation must be re-committed to acknowledge the new IP address of the switch.

Bridge Switch Functionality1. Establish which pair of switches will be the root and the backup root of the new com-

bined network.

a. The root and backup root should be the switch within the network that has the largest throughput capability.

b. The root and back-up root should have the lowest set of IP addresses among the network switches. (This is not mandatory but it is highly recommended.)

c. Any former root/backup switches will need the priority of their Bridge IDs changed.

Topology ConstantsWhen merging two or more networks, consider the physical layout of the topology. For example, if a four-layer Tree topology is to be combined with a Star topology, the Tree cannot be added to the Star. However, the Star can be added to the Tree but only at the root or layer one switches. Adding the Tree to the Star results in a five-layer Tree, exceeding the limit of seven switches (hops) between any two devices on the network. Adding the Star to the Tree's root or at the layer two switches only results in a broader Tree which does not exceed the seven hop limit.

There are an unlimited number of ways in which network topologies can be combined. However, the rules established in this document must be followed. Because of the unlimited ways of com-bining networks, take each case with its own considerations. Although this document cannot cover every possibility, the following are a few methods in which networks should be combined.

92

5. Combining Two or More MESH Control Networks B0700AZ – Rev L

Combining Star Network TopologiesIn Figure 5-1 below, A1 and B1 are root switches with the A2 and B2 as backup root switches within their respective Star topology networks. To combine the two Star networks with minimum impact to the physical layout of switch and cabling, the two Star networks are combined to form a three layered Tree topology, shown in Figure 5-2 below.

Figure 5-1. Combining Two Star Network Topologies Into One Tree Network Topology (Before)

Figure 5-2. Combining Two Star Network Topologies Into One Tree Network Topology (After)

To perform this, switch B2 must be taken offline, removed from the B network and configured as an edge switch in the A network. (Refer to The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA) for the definition of an edge switch.)

It is assumed that A1 and B1 are the root bridge switches within their respective networks and A2 and B2 are the back-up root bridges within their respective networks.

To do this, the following actions are required:

1. Verify that the A1 and A2 switches have a Bridge Priority value at least two (2) less than the B1 switch. If they do not, reconfigure the A1 and A2 switches with a lower priority value.

2. Power down the B2 switch and physically remove it from the B network, removing all cables attached to this switch.

A1 A2Star

B1 B2Star

A1 A2

B1 B2

93


3. The B2 switch must be re-configured as an edge switch in the A network. (Refer to The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA) for configuration information.)

4. With the B2 switch powered down, physically connect the B2 switch to the A1 and A2 switches via uplink (Trunk) cables one for each A root switch. The A root switch uplink (Trunk) port must be configured to accept the new cables (if previously dis-abled or if 100Mb ports are to be used).

5. Remove the previously cross-linked cables between the B1 and B2 switches from the B1 switch. Reconnect the B network edge switches to the B2 switch.

6. Power up the B2 switch. When this is completed, the B1 switch will re-span, releasing its root bridge function to the A1 switch. All devices on the B network have now been migrated to the A network.

7. Power down the B1 switch and physically remove it from what is now the A network, removing all cables attached to this switch.

8. The B1 switch must now be re-configured as an edge switch in the A network. (Refer to The MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA) for configuration information.)

NOTEIf the two networks are to be separated for maintenance at some point, insert a lower bridge priority value (i.e. 32000) for this B1 switch. At the time of separation, the switch will re-establish its root functions. The same can be done for the B2 switch as well, but with a higher bridge priority value (i.e. 32001).

9. The B1 switch must be added to the A network via uplink (Trunk) cables - one for each A root switch. The A root switch uplink (Trunk) ports must be configured to accept the new cables (if previously disabled or if 100Mb ports are to be used).

10. Make the physical connections to the A root switches utilizing the old cross-linked root connections to the B2 switch.

11. Reconnect the B1 switch to the former B network edge switches.

12. Power up the B1 switch.

You have just converted two Star network topologies into one Tree network topology.

If two Star network topologies are to be combined and it is desired that a Star topology be main-tained (see Figure 5-7 on page 97), you must also apply the process of eliminating the seconded pair of root switches, discussed in “Combining Tree Network Topologies” on page 96.

94


Combining Star with Tree Network TopologyThe process for combining a Star network Topology with a Tree network Topology is the same as for the previous procedure, “Combining Star Network Topologies” on page 93. See Figure 5-3 and Figure 5-4 below.

Figure 5-3. Combining a Star Network Topology with a Tree Network Topology Into One TreeNetwork Topology (Before)

Figure 5-4. Combining a Star Network Topology with a Tree Network Topology Into One Tree Network Topology (After)

A1 A2Tree

B1 B2Star

A1 A2

B1B2

95


Combining Tree Network TopologiesThe process for combining two Tree network topologies is the same as for the above two Star net-work procedures. Combining two Tree network topologies is possible if one or more of the topol-ogies has less than four tiers (layers). If two or more Tree network topologies has four tiers, then this combination process is not possible. A tier (layer) must be removed from one of the topolo-gies, as only one topology can have four tiers. This will cause an increase in effort and a higher risk in having a network failure.

Figure 5-5. Combining Two Tree Network Topologies Into One Tree Network Topology (Before)

Figure 5-6. Combining Two Tree Network Topologies Into One Tree Network Topology (After)

For two four-tier topologies, the root and back-up root switches of the second network must be removed (eliminated) and the second tier switches of this network must be connected to the root and back-up root of the first topology (see Figure 5-7).

A2Tree

B1 B2Tree

A1

A1 A2

B1B2

96


Figure 5-7. Combining Two (4) Tier Tree Network Topologies Into One (4) Tier Tree Network Topology (After)

Combining Ring Network TopologiesRing topologies are not recommend to be combined since the seven hop limit restricts the size of the network topology to a maximum of seven switches in the network Ring topology. When com-bining Ring topologies, it is recommended that the Ring topology migrate to the Star or Tree topology that best suites the physical layout of the site requirements. Because of the potential excessive amount of re-cabling, it is recommended that a plant shutdown be considered. Even combining a Ring with a Star or Tree topology, it is recommended that at least the Ring topology network side be shut down.

A Ring topology can migrate to a Star topology before combining the networks without requiring a plant shutdown. To migrate from a Ring topology to a Star topology, first determine which switches are the root and backup switches. If a root and backup are not configured yet, they should be configured first. A Star topology can be created by manipulating the uplink “Trunk” cables as shown in Figure 5-8 below. If there are no extra 1Gb uplink ports, the 100Mb uplink ports of the B1/B2 switches can be used. Ideally, if two rings are to be combined, the final root and backup root switches should be replaced with C2 Series or N-Series switches so that all uplinks utilize 1Gb connections.

By using Figure 5-8 below and Figure 5-7 above (which combines two stars), two rings could be converted into stars and then combined.

A1 A2 B1 B2

97


Figure 5-8. Combining Ring Network Topologies

Move cables one at a time

B1 B2 Root and Backup

B1 B2

B1 B2

Added cables

98

Appendix A. COMEX Fault Handling on The MESH Control Network
This chapter provides information on Communication Executive (COMEX) fault handling in The MESH control network.

COMEX Layers

COMEX Applications LayerThis layer provides connectionless and exposed LLC communications services. Connectionless messages are contained in unit data packets.

There are 2 types of connectionless messages - acknowledged and unacknowledged. For acknowl-edged communications, there is a 2-way acknowledgement (handshake), where the receiver of the unit data packet replies with an ack1 packet to the initial sender, which then responds with an ack2 packet.

No replies (acks) are returned for unacknowledged message types. However, these message packets are sent twice, first out the primary port, and then 1 second later, out the secondary port. The Invensys multicasts are unacknowledged connectionless messages. The Invensys “Broadcast” is actually one of the Invensys Multicasts.

Cable selection is based on an array in COMEX. The number of transmits is used as an index into this table to select the cable. Seven (7) transmission attempts (Versatile Real-Time Executive (VRTX) devices) are allowed, with the number of transmits starting at 1. Therefore, the first entry in the array (index 0) is a filler and not used. The remaining entries indicate the cable selection as follows: Primary, Primary, Secondary, Secondary, Primary, Primary, and Secondary. On the third transmission attempt, a cable test is requested before attempting to send the message. When the number of transmits exceeds the maximum (7 for VRTX devices), the message is flushed. For the I/A Series MESH and 7.x Control Network stations (Windows XP® workstations and Nucleus Plus devices), this maximum is also 7.

Application Layer Protocol and TimersA transmitted acknowledged connectionless Unit Data packet requires an Ack1 response from the receiver. Upon receiving the Ack1 response, the initiator responds with an Ack2 packet.

A transmitted unacknowledged connectionless Unit Data packet is directed to the Primary cable, and requires no response. Rather, after waiting 1 second, the packet is re-directed to the Second-ary cable. (Since the Data Link Layer of the I/A Series MESH and 7.x Control Network systems determines the active communications port, the above cable directions are ignored.) A reference delay is used to time the receipt of Ack1's and Ack2's and the duplicate transmission of unac-knowledged packets. This time delays are as follows:

♦ round trip time delay - 1.01 seconds

99

B0700AZ – Rev L Appendix A. COMEX Fault Handling on The MESH Control

This is a basic delay period for stations on the same Node, for which some other delays are based. For stations on another Node on the same LAN, 1 is added to this value. For stations on another LAN on the same Site, 2 is added to this value. Note that the Site in the comms hierarchy was never implemented.

♦ acknowledged UD packet transmitted (retransmit time-out) - 1 second time-out for receipt of Ack1 response

Failure to receive response within the time-out period results in the UD packet being re-transmitted, up to the limit of retries. This value is subject to ongoing optimization in The MESH network devices, and may change without notice.

♦ Ack1 response sent - Time-out period is taken from the original acknowledged UD packet (its retransmit time-out) and modified, for receiving the Ack2 response.

The time-out period is multiplied by the maximum number of connectionless retries allowed and 1 added. Then, the round trip delay is added to the time - 1.01 seconds for same node, 2.01 seconds for same LAN, and 3.01 seconds for same site. There-fore, the reference delay is between 9.01 and 11.01 seconds, for VRTX stations. (Because, the site hierarchy part of the communications system was never implemented, the same LAN would be the furthest round trip delay - 10.01 seconds.) Failure to receive the Ack2 response is the same as if the Ack2 was received - resources are released.

♦ unacknowledged UD packet transmitted - 1 second time-out.

Timer expiration results in sending duplicate unacknowledged UD packet on other cable. This is always the case, since no response is expected that would stop the timer.

♦ unacknowledged UD packet received

The time-out period is taken from the original unacknowledged UD packet and mul-tiplied by 2. Then, the round trip delay is added to the time - 1.01 seconds for same node, 2.01 seconds for same LAN, and 3.01 seconds for same site. Therefore, the ref-erence delay is between 3.01 and 5.01 seconds. (Because, the site hierarchy part of the communications system was never implemented, the same LAN would be the furthest round trip delay - 4.01 seconds.) When the duplicate packet is received, or with timer expiration, receive resources are released.

Transport LayerThis layer provides connected communications services. The protocol of this layer is based on a reduced subset of what appears to be class 4, as defined in RFC 905, “ISO Transport Protocol Spec-ification, ISO DP 8073”, dated April 1984. The message is contained in a data transfer packet (DT). Connections are set up by sending a Connection Request packet (CR). The response is a Connection Confirm packet (CC) to accept the connection, or a Disconnect Request (DR) to refuse the connection. When the initiator receives an acceptance (CC packet), an Ack (AK) packet is sent to acknowledge the connection. DT packets are acknowledged with an AK packet. In addition, if either side of the connection has not sent a packet in a certain time-out period, an AK packet is then sent as a “heartbeat” to show that the connection is still active. Disconnects are initiated with a DR packet. The disconnect is accepted by replying with a Disconnect Confirm (DC) packet.

DT packets are sequenced and use a “window” controlling the number of outstanding DT pack-ets before an AK is received. The sequence number starts at 0 and is incremented for successive DT packets. The receiver of the DT packet responds with an AK, indicating the sequence number of the next expected packet. If a DT packet is received in error, then the AK will contain the sequence number of that packet. A receive credit value is given in the CR, CC, and AK packets,

100

Appendix A. COMEX Fault Handling on The MESH Control Network B0700AZ – Rev L

indicating the size of the “window” for the receiver. In effect, the transmitter can have no more than the receive credit number of transmitted packets outstanding with the receiver.

When a connection is requested, (or any packet expecting an ack response) the initiator will wait the Retransmit Timer period for the response, before retransmitting the packet. When a connec-tion is established, DT packets or heartbeats (AK's) will be sent every Window Timer period. A DT or AK packet is expected to be received within the Inactivity Timer period, in order to main-tain the connection. When a DT packet is received, the receiver will wait no longer than the AK Timer period before transmitting a response (AK).

Messages could be received in multiple data packets. This is called a multi-packet receive. The packet indicates if it is the last packet of a transmission (EOT bit of the sequence number octet). Multi-packets need to be linked into a single message before being delivered to IPC. In addition, during multi-packet receives, the receive credit get boosted above normal and an AK set to change the credits with the sender. With the last packet, the receive credit gets set back to normal, and an AK is then sent to change the credit back to normal with the sender. (At the sender, it becomes his transmit credit.)

When a connection exists, all messages to be transmitted are placed in the Wait Ack Queue (WAKQ) to wait for their acknowledgment. Received packets (DTs or ACKs) that are out of sequence with what is expected are handled according to the following table:

Transmissions expecting a response are attempted 8 times (7 retries), before the packet is dis-carded. For the I/A Series MESH and 7.x Control Network stations, this limit is 8 attempts for both Windows XP workstations and Nucleus Plus devices (7 retries). Transmissions not expecting a response (AK and DC packets) are only sent once. The cable selection is governed by an array.

Table A-1. Out of Sequence DT and Ack Packets

Received Packet Action Retransmit Timer Ack Timer

ACK seq < outstand-ing DT pkt's last seq.

Acknowledged packets removed from WAKQ, if present. Available new packets sent up to xmit credit limit.

Restarted if any packets on WAKQ acknowl-edged and new ones sent. If WAKQ empty, timer stopped.

-

ACK seq same as last ACK seq received.

Ignored. No action. -

ACK seq > outstand-ing pkts last seq + 1

Out of sequence counter incremented. ACK ignored.

No action. -

DT seq < next expected - 1.

Dumped. - Start timer if not already running.

DT seq = next expected - 1

Dumped and ACK sent.

- (This packet was the last one acknowl-edged and timer should already be stopped.)

DT seq > next expected

Dumped. - No action.

101


The number of retries is used as an index into the array. Cables in this array are selected as follows: primary, primary, secondary, secondary, primary, secondary, primary, secondary. Packets expecting a response will use this array. Packets not expecting a response (AK and DC), use another array. The result of using this array, is that the cable is swapped between the primary and secondary, with every transmit. Because AK and DC packets use private resources that is never released (these packets are created in the TL), the transmit count will keep incrementing, so that these packets alternate cables.

The retry count is for an individual message (not a response) on a given connected channel. The transmit count is only for responses being sent using the private resources. Individual messages are retried. Responses (AK, DC) are not retried, because they do not get a response.

When the retry count is 2, a cable test is requested. When the retry count exceeds the maximum, the message is flushed.

Transport Layer Protocol and TimersThe timers used by the Transport Layer are:

♦ Retransmit Timer - 0.5 seconds for VRTX devices (for the I/A Series MESH and 7.x Control Networks. this value is 0.5 seconds for Nucleus Plus devices and 0.5 seconds for Windows XP workstations)

Maximum time allowed between sending a packet and receiving an AK for it. This timer automatically restarts itself. Timer expiration causes a transmission reattempt of all packets in the Wait Ack Queue, up to the retry limit. When outstanding packets are AK'ed, this timer is stopped (and packets removed from the queue).

♦ Inactivity Timer - 10 seconds

Maximum time allowed to receive heartbeat AK's (or DT's). This is a one-shot timer. It is restarted whenever a packet is received. At timer expiration, the connection is aborted.

♦ Window Timer - 2 seconds for VRTX devices (for the I/A Series MESH and 7.x Con-trol Networks, this value is 2 seconds for Nucleus Plus devices and 2 seconds for Windows XP workstations) Maximum time allowed between sending heartbeat AK's (or DT's)

This timer automatically restarts itself. It is also restarted whenever an AK or DT is sent. At expiration, an AK packet is sent.

♦ AK Timer - 75 msecs

Maximum time allowed between the first DT received and transmitting its AK. This is a one-shot timer. At expiration, an AK packet is sent.

Normally, when the number of received DT packets equals the limit, an AK packet is sent. In this case, the timer provides a cap on the time between received data and its AK transmission. The limit is 1.

When a connection is requested with a CR packet, a CC packet response is generated by the receiving end. The Retransmit Timer is used to time the response. If the timer expires before get-ting the response, the CR packet is resent, up to the maximum retries, before the packet and the connection are aborted. If the protocol is successful, a connected channel is created. Then the Inactivity Timer is used by both ends of the channel for receiving packets from the other end of the channel. These packets can be either DT packets or Acks (DT acknowledgements or heart-beats). If the Inactivity Timer expires, the connection is aborted.

102

Appendix A. COMEX Fault Handling on The MESH Control Network B0700AZ – Rev L

With a connection established, and a DT packet sent, an Ack response is expected. The Retrans-mit Timer is used to time the response. If the timer expires before getting the response, the DT packet is re-sent, up to the maximum retries, before the packet and the connection are aborted.

With a connection established, and a DR packet sent, a DC response is expected. The Retransmit Timer is used to time the response. If the timer expires before getting the response, the DR packet is re-sent, up to the maximum retries, before the packet and the connection are aborted.

An aborted connection causes a DR packet to be sent. The expected response and timers are the same as previously described. The difference is that instead of IPC requesting the disconnect, the request is initiated by the Transport Layer.

Network LayerThe Network Layer provides routing to the Nodebus, to another Nodebus (through the LAN Interface), and back into the box (upper layers).

OperationThere are no states in the Network Layer operation, because no protocol is performed. The pri-mary responsibility of this layer is routing the packets. Packets are routed to the local node, to the LAN Interface in VRTX devices for other nodes, and back to the upper layers in the same station. In The MESH network devices, off-node packets are routed to a specific Address Translation Sta-tion (ATS), rather than to a LAN Interface.

Point-to-point packets are routed to the upper layers when the destination NSAP (site, LAN, and node ID's) is the same as the source NSAP. The destination and source are on the same station and the packet is routed to the upper layers as if it was a received packet.

Multicast packets are always sent out the station. They are also routed to the upper layers of the source station, if the Reroute bit (in the packet) is set and the multicast group is active in the sta-tion.

Logical Link Control LayerThe implementation of this layer is based on IEEE 802.2, class 1 (Type 1 Operation only), using only unnumbered UI commands (information). No other U commands are supported. There-fore, the poll/final bit is not used (it is not valid for UI commands).

Transmit OperationThere are no states used in an LLC transmit operation. The LLC header is simply added to the packet and it is passed on to the MAC layer.

Receive OperationThe LLC distributes received packets according to whether they are normal data packets, diagnos-tic packets, or exposed LLC packets. Several additional packet types are defined for The MESH network devices.

The routing of connected and connectionless packets is based on the Network Layer destination LSAP (Link Service Access Point). These packets specify network routing (Network Layer LSAP), providing for routing by the destination LLC Layer to the destination Network Layer.

For exposed LLC packets, 2 LSAPs are supported. The Application Layer 802.1 individual LSAP is used for downloading station images, and the Application Layer 802.1 group LSAP is used for

103


cable test messages (diagnostics). These packets are distributed directly to the Application Layer.

The diagnostic packet used for a cable test is sent by NFD with a Station Manager destination LSAP, via exposed llc (through IPC and the Application Layer). Received Station Manager packets are passed by the LLC Layer directly to SMD (station manager diagnostics), which echoes them back directly through the LLC Layer, as an Application Layer 802.1 group destination LSAP. These are then passed by the receiver LLC Layer directly to the Application Layer (and thus, back to NFD).

The MESH network devices provide for additional destination LSAPs:

♦ NFD PROXY for The MESH network to Nodebus NFD cable test proxy packet.

♦ MAC QUERY for a Mac Query packet.

(Other LSAPs are used by The MESH network comms drivers and ATS devices for validating net-work integrity.)

MAC LayerThe MAC layer is designed for the individual hardware for which it interfaces.

104

Appendix B. The MESH Network Fault Handling
This chapter provides information on fault handling in The MESH control network.

Fault handling on The MESH network is handled on a per station basis. This error handling pro-cess is protected by a patent. This description applies to all dual port COMEX stations which have two roles for their ports, including Workstations, FCP, ZCP, and ATS Stations:

♦ an active port based on the system address for that station.

♦ a standby port that only is used for testing with a different MAC address until it is determined that the active port is not functioning correctly.

COMEX implements seven retries on the active port spaced over 3.5 seconds for all connected messages. Connectionless acknowledged messages perform seven retries over seven seconds. COMEX relies on lower level mechanisms to select the best possible port as the active port. The port selecting behavior of the stations is as follows.

LINKThe first line of defense is the Link Integrity Signal. This signal is derived by the PHY Interface hardware and is indicative of the presence of “IDLE” symbols received. If a port does not have “LINK”, it cannot send or receive. If the Link signal goes inactive, the port is not usable and the driver will switch the station address on the module to the other port if it does have a “LINK” sig-nal. Typically switches stop sending if they detect no activity on a port (to conserve power) so link becomes a bi-directional indication. Loss of LINK switchover will be indicated to the customer as PORT A (or B) failure in the smon_log and SMDH.

Causes of loss of link:

1. Switch power down

2. Switch failure

3. Cable failure of either the transmit or receive fiber

4. Dirty cable

5. Module hardware failure

“PORT TEST” PacketsIf both LINK signals are active two “PORT TEST” packet are sent by the driver every 300 milli-seconds, one “PORT TEST” packet is sent from the “A” port to the “B” port and one “PORT TEST” packet from the “B” to the “A”. In the body of the packet is the text “Prim Port Test” and “Alt port Test”. Reception of these packets in both directions indicate that both ports have equal connectivity to the rest of the network. As long as “LINK” is good and the “PORT TEST” pack-ets succeed, no port switching will occur.

105

B0700AZ – Rev L Appendix B. The MESH Network Fault Handling

A single port test packet failure is flagged and the Excessive Collision Counter is incremented but no switchover occurs until two “PORT TEST” in a row have experienced a failure. A “PORT TEST” failure is defined as a packet in either direction failing.

If two successive “PORT TEST” tests fail due to the inability to receive both packets the module will initiate the “PING” test. The failure of this test indicates that there is a fault in The MESH which prevents normal communications. The port test packets will cease for 400 milliseconds while the “PING” test attempts to detect the best way to survive the problem. After the llc_ping test runs the module goes back to sending port tests until the network is healthy.

Causes of “PORT TEST” failure:

1. Switch failure causing islanded switches

2. Message storm substantially interfering with normal communications

3. Internal switch failure of the switch connected to the “A” port or the “B” port

4. Module hardware failure

5. Broken or damaged cable

“LLC_PING” TestThis test involves sending a “PING” packet to up to seven stations that have previously been in communication with this controller. It is used to allow the module to select the port with the highest connectivity with its most recent COMEX partners based on a most recently received table. This IS NOT the standard IP “PING” packet instead it is a COMEX llc_ping type(DSAP = 0x0A, SSAP = 0x0A). Only COMEX stations are sent this message to guarantee that the targeted station is running the SW that will reply to this “PING”. The packet indicates that this is a llc_ping request and is sent on both the “A” port and the “B” port. When any COMEX sta-tion receives this request it responds with a LLC_PING_response packet with the same format(DSAP = 0x0A, SSAP = 0x0A) to the senders MAC address. The station who initiated the llc_ping message then tallies the responses and make a decision whether to switch ports or not.

1. IF all responses are received on both cables this is a “Good” case and no switchover is performed and no messages are sent.

2. IF more responses (for example 4 responses on alternate and 2 on Primary) are received on the Alternate port a port switch is performed (no Hot-Remarry should occur on FT stations). This port switch places the Alternate as the Primary port and re-assigns the MAC addresses of the ports. A broadcast is sent to update all switches with the new MAC address location. A message is sent to the alarm printer and the smon_log if enabled.PING Test Sent = 7 Replies = Pri 2 Alt 4 new cable = B (or A)

NOTEA workstation does not currently send any messages to the alarm printer/smon_log. Workstations run in promiscuous mode. Upon a Primary port swap, the NIC is not reprogrammed but the software does use the Primary port Mac address on the Alter-nate port when a swap occurs.

3. IF AN EQUAL number of responses is received and the number is less than the total sent a one-time switchover is performed. This switchover is to resolve situations with

106

Appendix B. The MESH Network Fault Handling B0700AZ – Rev L

an equal number of local and remote stations causing islanding. It is done to cascade all stations over to a healthy path. No more switchovers on “tie” situations are allowed until successful “A” to “B” port tests are detected indicating a return to healthy MESH. A message is sent to the alarm printer PING Test Sent = 7 Replies = Pri 3 Alt 3 new cable = B (or A)

4. IF more responses are received on the Primary port, no switching is performed and a “PORT FAILURE” message for the Alternate port is sent to the alarm printer.

ZCP-FCM CommunicationsAll communications between the ZCP and the FCM use raw IP packets. The ZCP sends an alternating request/reply transaction on each Ethernet cable to talk to the FCM pair at all times. On discovery of any failure, the field connection has its reliability score reduced. The ZCP uses the more reliable cable exclusively while background mechanisms try to rehabilitate the failed cable.

The ZCP maintains a quality of connection for each FCM separately. If one FCM has a problem on channel A, that FCM will prefer to use channel B. All others would still use A or B normally.

There are no retries. All Ethernet, IP, and port addresses are unique to the particular network.

NOTEIf the user has required the ZCP to use the “A” Ethernet cable and the “A” cable is in fact failed, the ZCP will blindly follow the requirement to use the “A” cable even though it does not work. This is done to aid debugging of problems.

NOTETwo FCMs can have the same addresses if they are on separate networks; for exam-ple, behind different ZCPs and not on The MESH.

There is no LLC3 protocol on the I/O side between the ZCP and the FCM.

ZCP and FCM communication behaves as discussed below in the following cases:

Case 1: The ZCP fails to receive a response on the primary port to an FCM and/or switch.

Behavior: The ZCP retains a health score on each of its connections. It maintains com-munications on the “A” or “B” bus, whichever is more healthy. However, the FCM is also trying to recover the bad cable, and is sending health packets on the bad cable. A success-fully delivered packet raises the ZCP’s health score, so the bad cable’s health score may increase. Multiple successes raise the cable’s health score to completely good. When the score is equal that of the other cable, the ZCP will start to alternate between cables again for its communication to the FCM.

Case 2: The ZCP communicates with several FCMs and does not receive a response on a port.

Behavior: The ZCP maintains a quality of connection for each FCM separately. If one FCM has a problem on the “A” bus, that FCM will switch to the “B” bus. All other FCMs

107


would still use “A” or “B” normally. A message indicating this change should be generated by the ZCP on the FCM's behalf.

DIAGNOSTIC InformationDiagnosing MESH network failures is complicated but here is some general information

1. PORT FAILURE MESSAGE

a. Most likely caused by loss of the LINK signal.

b. Most likely caused by a cable problem or dirty (very common) Fiber connections.

c. Loss of link may be reported by both the switch and the station “PORT FAIL-URE” message and the SMDH port field.

d. Port Failure is also indicated by a persistent path failure from the Primary port to the Alternate port. The Alternate port will display the port failure message.

2. FT LINK Problems

a. On FT station both stations must have a common link on at least one cable to stay married.

b. If the Primary loses both links it will reboot and the Shadow will become Primary.

c. Modules will not marry without at least one common link. If no common link is found a module will go RED/GREEN.

3. ZCP LINK problems

a. For ZCPs link failure on the Control network are handled in the same fashion as failure on the Application network.

b. For ZCP I/O functionality as of 8.2 there must be a path from the “A” port of the ZCP to the “A” port of the FCM or the “B” port to “B” port in order for the ZCP to maintain comms. The ZCP cannot talk from the “A” port to the FCM “B” port MAC address. This means that cross-wired systems have multiple single points of failures as follows:

♦ ZCP port “A” to Switch 1 port 1

♦ ZCP port “B” to Switch 2 port 1

♦ FCM port “A” to Switch 2 port 2

♦ FCM port “B” to Switch 1 port 2

♦ Switch 1 port 3 to Switch 2 port 3

If this configuration was built the ZCP to FCM communications would work fine with all hardware functional. Any of the following failures would cause loss of I/O communications:

♦ Switch 1 -power down or total failure

♦ Switch 2 -power down or total failure

♦ Switch 1 port 3 or switch 2 port 3

♦ Cable Switch 1 port 3 to Switch 2 port 3

4. SMDH counters

a. MAC resets indicate either Hot-Remarries or Port problems.

108

Appendix B. The MESH Network Fault Handling B0700AZ – Rev L

b. Excessive Collisions indicate either continuous (evenly incrementing approxi-mately 1 per second) or intermittent MESH problems (incrementing at less than 1 per second) causing “A” to “B” failures.

c. At the present time, workstations do not indicate “A” to “B” port test failures unless a port switch occurs.

109


110

Index
CCable

fiber optic 60, 61specifications 59twisted-pair 66

Cablingfiber optic 63

Cleaningfiber optic cable 87

DDisplay, Switch Configuration Information 80Display, Switch Domain 81Display, Switch Port Equipment Change 83Display, Switch Port Equipment Information 84Display, Switch Ports 82Display, Switched Equipment Information 78Display, Switched Network 76, 77

EEthernet switches

connecting 67indicators 85, 87

FFiber optic cable 18, 60, 61

cleaning 87handling 85

Fiber optic cabling 63

GGigabit Ethernet uplink ports 68

HHandling

Fiber optic cable 85

II/O network 12

design rules 36

111

B0700AZ – Rev L Index

illustration 14topology configuration 58

IndicatorsEthernet switches 85, 87media converter 85

IOC (I/O Controller) 12

MMultimode fiber. See Fiber optic cable, multimode

NNetwork components connections 67Network topology 36Null hub 18, 66

PPrefabricated cables 66

SSCAS - Switch Configurator Application Software 17Single mode fiber. See Fiber optic cable, single modeSite planning 23SMDH 74Switch Configuration Information display 80Switch Domain display 81Switch Equipment Information display 78Switch Port Equipment Change display 83Switch Port Equipment Information display 84Switch Ports display 82Switched Network display 76, 77System Management displays 74

TTopology, network 36Troubleshooting 72Twisted-pair cable 66

UUplink port 68

VVLANs 3, 23, 30

112

Index B0700AZ – Rev L

113

B0700AZ – Rev L Index

Printed in U.S.A. 1009

IPS Corporate Headquarters5601 Granite Parkway Suite 1000Plano, TX 75024United States of Americawww.ips.invensys.com

Foxboro Global Client SupportInside U.S.: 1-866-746-6477Outside U.S.: 1-508-549-2424 or contact your local Foxboro representative.Facsimile: 1-508-549-4999