1 Computer Science, University of Warwick Computer Science, University of Warwick Cluster Networks Cluster Networks Introduction Communication has significant impact on application performance. Interconnection networks therefore have a vital role in cluster systems. As usual, the driver is performance… An increase in compute power typically demands proportional increases in lower latency / higher bandwidth communication services.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1Computer Science, University of WarwickComputer Science, University of Warwick
Cluster NetworksCluster Networks
IntroductionCommunication has significant impact on application performance.
Interconnection networks therefore have a vital role in cluster systems.
As usual, the driver is performance…An increase in compute power typically demands proportional increases in lower latency / higher bandwidth communication services.
2Computer Science, University of WarwickComputer Science, University of Warwick
Cluster NetworksCluster Networks
Issues with cluster interconnections are similar to those with normal networks:
Latency & Bandwidth
Topology type (bus, ring, torus, hypercube etc).
Routing
Direct connections (point-to-point) or indirect connections.
NIC (Network Interface Card) capabilities.
Physical medium (Twisted pair, fibre optic)
Balance between performance and cost
3Computer Science, University of WarwickComputer Science, University of Warwick
In standard LANs we have two general structures:Shared network (bus)
• All messages are broadcast… each processor listens to every message.
• Requires complex access control (e.g. CSMA/CD).
CSMA/CD: Carriers Sense Multiple Access with Collision Detection
• Collisions can occur: requires back-off policies and retransmissions.
• Suitable when the offered load is low - inappropriate for high performance applications.
• Very little reason to use this form of network today.
Switched network
• Permits point-to-point communications between sender & receiver.
• Fast internal transport provides high aggregate bandwidth.
• Multiple messages are sent simultaneously.
4Computer Science, University of WarwickComputer Science, University of Warwick
Metrics to evaluate network topologyMetrics to evaluate network topology
Useful metrics for switched network topology:Scalability : the network’s switch scalability with nodes.
Degree: number of links to / from a node
measure aggregate bandwidth
Diameter: the shortest path between the furthest nodes.
measure latency
Bisection width: the minimum number of links that must be cut in order to divide the topology into two independent networks ofthe same size (+/- one node). Essentially a measure of bottleneck bandwidth - if higher, the network will perform better under load.
5Computer Science, University of WarwickComputer Science, University of Warwick
12Computer Science, University of WarwickComputer Science, University of Warwick
SwitchingSwitching
Operational modes:Store-and-forward:• Each switch receives an entire packet before it forwards it onto the next
switch - useful in a non-dedicated environment (I.e. a LAN). • usually, there is a finite buffer size so it is possible that packets will be
dropped under heavy load. • Also impose a larger in-switch latency.• Can detect errors in the packets
Worm hole routing (Also called cut-through switching): • Packet is divided into small “flits” (flow control digits).• Switch examines the first flit (header) which contains the destination
address, sets up a circuit and forwards the flit immediately.• Subsequent flits of the message are forwarded as they arrive (near wire-
speed).• Reduces latency and buffer overhead.• Messaging occurs at a speed close to the processors being directly
connected.• Less error detection
1Computer Science, University of WarwickComputer Science, University of Warwick
2Computer Science, University of WarwickComputer Science, University of Warwick
3Computer Science, University of WarwickComputer Science, University of Warwick
1Computer Science, University of WarwickComputer Science, University of Warwick
Cluster Network ProductsCluster Network Products
Cluster interconnects include, among others:
Gigabit Ethernet
Myrinet
Quadrics
InfiniBand
1Computer Science, University of WarwickComputer Science, University of Warwick
2Computer Science, University of WarwickComputer Science, University of Warwick
Interconnects in Top500 list Interconnects in Top500 list –– 11/200911/2009
3Computer Science, University of WarwickComputer Science, University of Warwick
Interconnects in Top500 list Interconnects in Top500 list –– 11/200811/2008
4Computer Science, University of WarwickComputer Science, University of Warwick
Gigabit Ethernet:The technology has matured and now offers very good performance at a very low cost.
Latency performance is moderate - many Ethernet switches are designed for general LANs (store & forward) where latency reduction is not necessary the primary incentive (the latency isorder of ms).
Zero-copy OS-bypass message passing can be supported with programmable NIC and direct memory access.
5Computer Science, University of WarwickComputer Science, University of Warwick