Top Banner
Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006
36

Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Dec 26, 2015

Download

Documents

Sophie Webb
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 1

QsNetIII and QsTenG

Networks for High PerformanceComputing

12th June 2006

Page 2: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 2

AGENDA

Company introduction

QsTenG Roadmap

QsNetIII Roadmap

Page 3: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 3

COMPANY

Page 4: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 4

Summary

Established in 1996, with locations in the US, UK, Italy and China

Leading supplier and developer of high performance networking products and resource management software

Subsidiary of Alenia Aeronautica and wholly owned by Finmeccanica. One of the largest industrial group in Europe in the defense, energy and security industries.

Developing proprietary HPC network and industry standard network products (10 GigE first)

Strategic market expansion: investments in USA (Quadrics, Inc.), Asia Pacific and Europe.

Currently 50 employees (70 % engineers)

Page 5: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 5

HistoryQuadrics R&D Team out of Meiko

1996 Incorporation as UK Limited Company, part of the Finmeccanica Group

(SIFI.MI Italian Stock Exchange)

1998 Manufacture in US through Solectron

1999 Compaq OEM agreement

2000 Compaq Third Party Manufacturing Agreement (Royalty)

2001 Shipment Direct from US to US customers

2005 Incorporation of Quadrics, Inc.

Focus on High Performance Interconnects and Resource Cluster Management with

a mission to be the market leader in such solutions

Page 6: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 6

Org Chart

G ia co m o F a v illaC F O

J im C o rkeM a nu fa c tu ring

M o ra y M cL a renH a rd w a re

D u n ca n R o w e thS o ftw a re

M o ra y M cL a renR & D

R a im o n d o F o rlinH P - S a le s N o rth A m e rica

A n ton io V oxM a rke tin g & S a les

C ris to fo ro R o m a n e lliC E O

Page 7: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 7

2004 2005 2006 2007

96 PORTS SWITCH

STANDARD10 Gbit/s ETHERNET

Strategy - Roadmap

QsNetII

ELAN 4 & ELITE 4

PROPRIETARYQsNet

ELAN 5ELITE 5

QsNetIII

288 PORTSSWITCH

Page 8: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 8

QsTenG

Page 9: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 9

QsTenG - Overview

10 Gbit/s Ethernet layer 2 switch family

Data centre and clustering market

QS-TG108 switch: up to 96 ports, copper CX4

Low latency - 0.5s to 1.5 s port to port

Compact switch chassis (8U)

Can be used with standard 10 Gbps Ethernet NICs

Page 10: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 10

QS-TG108 Switch - Architecture 12 ports microchip building block

Two stage fat-tree topology

Scalable 8-96 ports

450ns per stage latency

Source routed

Building blocks

Backplane links

Computing nodes

CX4 copper cables

Page 11: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 11

QS-TG108 - Physical Design

8U chassis

1U PSU subsystem

Front to back airflow

All connections on rear of switch

Page 12: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 12

QS–TG108 - Rear View

1 to 12 line cards

(part ID QM601)

1 to 2 control cards

(part ID QM602)

Page 13: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 13

QM601 - 8 Ports Line Card

8 CX4 ports to front panel

4 CX4 ports to backplane

Control processor

Page 14: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 14

QM602 - Control Card

Master control processor

External control and management 1 Gbps Ethernet connection

12 CX4 backplane connections

Page 15: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 15

QS–TG108 - RAS Features

Redundant control architecture

1 from 2 (2 from 3) redundant power supply

Dual sequential fans

Page 16: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 16

QS-TG108 – Control Structure

Embedded Linux control processor for each switch card

Switch configuration Stats gathering

Control card – QM602

Single point of control SNMP switch agent Fail-over to controller on alternate QM602

Dual 1 Gbps Ethernet control interfaces

Page 17: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 17

QsTenG – Firmware

Embedded Linux control software

SNMP Device management support

Power on self test

Hardware diagnostics

Performance monitoring

Page 18: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 18

QsTenG – Software

MPI-2 common to QsNet and QsTenG

Adapter specific drivers

Quadrics Resource Management System (RMS)

24 7 production cluster management

Page 19: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 19

QsTenG – CX4 Adapters

QsTenG family of switches will operate with all standard 10 Gbps Ethernet adapters

Intel – validated Chelsio – validated Myricom – validated Neterion – validation about to start Neteffect – validation about to start

Page 20: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 20

Managed Layer 2 Protocol Support – Features

LVL7 FASTPATH® Software Certified protocol support for routing and management including

VLAN - Network grouping/segmentation- IEEE 802.3ac - VLAN Tagging- IEEE 802.1v - Protocol-based VLAN - IEEE 802.1Q - Virtual LAN with port based VLANs

Spanning Trees - Broadcast storm/loop avoidance- IEEE 802.1S – Multiple Spanning Tree- IEEE 802.1W - Rapid Spanning Tree- IEEE 802.1D - Spanning Tree

Flow Control- IEEE 802.3x - Flow control

Priorities- IEEE 802.1p — Ethernet Priority with User Provisioning &

Mapping

Page 21: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 21

Managed Layer 2 Protocol Support – Features Cont…

GARP - Attribute register/de-register IGMP - Snooping - Multicast membership Link aggregation - Channel bonding

- IEEE 802.3ad — Link Aggregation

Port mirroring - Copy for analysis Jumbo Ethernet Frames Power management for individual slots Fan control Redundancy (Cold Failover) Hot plug …

Page 22: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 22

Federation Support – Features

Federation of multiple Switches

Quadrics Internal Software development

Federation Support

Support for 2000 ports

Fat Tree Topology

Page 23: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 23

QsTenG – Summary

A new generation of 10 Gbps Ethernet switch for new classes of bandwidth hungry applications.

Takes Quadrics scalable switch design into the standards based interconnect market.

Page 24: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 24

QsNetIII

Page 25: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 25

QsNetIII – Overview

New host adapter – based on Elan5 microchip

PCI Express 16x, DDR

New custom fabric – based on Elite5 microchip

High radix switches DDR CX4 links

Common mechanics and control structure

128 node building block (8U)

Page 26: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 26

Elan5 – Overview (product ID QM700)

2 QsNetIII link

Multiple Thread processors

Local memory (1Mbytes) Buffer space, routes, page

tables Memory Management

4 pages sizes 4M-32G TLB (64 16 pages)

Host Interface PCI-Express 16x DDR support

Fabric

Bridge

x8

PCIe 16 Lanes

PLL

External Code Interface

Clocks

16K x 8 x 8 banks = 1MB ECC RAM

Host I/F

PCIe

TLB Cmd Launch

Ext i/f

X2

CX4/ QsNetIII

Link

SERDES

SAC/SA SIP SOP

UAC/UA UOP UIP

Slave Access Ports (slave only)

Unit Access Ports (master/slave)

Addr Translator

Buff Mngr Host Cache

Free List

Local Memory

X2

CX4/ QsNetIII

Link

ThreadProc I$ 16K

DMEM 8K

ThreadProc I$ 16K

DMEM 8K

ThreadProc I$ 16K

DMEM 8K

ThreadProc I$ 16K

DMEM 8K

ThreadProc I$ 16K

DMEM 8K

ThreadProc I$ 16K

DMEM 8K

ThreadProc I$ 16K

DMEM 8K

Page 27: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 27

Elan5 – Thread Processor

Two thread instructions per cycle 500-600 MHz clock 128 registers 16K instruction cache 8K data buffers

- Assembling network packets- Checksum and CRC generators on write port

DMA read, write, set event, wait event

Page 28: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 28

Elite5 - Overview

16 x4, 32 x2 switch component

Or multiple 8,4,2,1 switches

Physical layer DDR XAUI

4 x 6.25Gbit/s 2.5Gbytes/s peak in each direction

Broadcast & barrier support

32 x 32 x 2 core

Page 29: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 29

Elite 5 – Switch Packaging, cards

16 ports link card (Part ID QM701)

Building block for: 256 ports stand alone switch 128 ports to computing nodes, 128 ports

to top level switches chassis

Top switch card (Part ID QM702)

Building block for: 256 ports stand alone switch Federated networks

Top switch card (Part ID QM703) Federated networks

Page 30: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 30

QsNetIII – 128 Ports chassis

Building block for federated networks

8 1024-way systems 16 2048-way systems x32 4096-way systems x64 8192-way systems

128 ports to nodes, 128 backplane links to top level switching cards

2 stage fat tree

9U high

Page 31: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 31

QsNetIII – 256 Ports chassis

High Density stand alone switch

800 Gbyte/sec bi-section bandwidth

2 stage fat tree

14U high

Can be used to provide multiple 32 and 64 top switches for very large systems

Page 32: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 32

4096 Way System – 1U Nodes

128 node scalable units

Each scalable unit has a node level switch

Upgradable bisectional bandwidths:

1/8; ¼; ½; full

Page 33: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 33

QsNetIII – 4096 Port Network Topology

Page 34: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 34

4096 1U SYSTEM - FLOOR PLAN

System size: 10m 20m (33' 66') with an aisle spacing of 5'. Maximum cable length is approximately 13m (43')

Page 35: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 35

Possible Lustre configuration with 1/8 bisectional bandwidth

48 Lustre storage servers (2 meta dataservers, 2 management nodes and a small number of login nodes).

Divided equally between the first and last node switches.

Should be configured in pairs (with one server from each pair attached to each node switch).

This configuration will maximise the Lustre I/O bandwidth with sparse connectivity between the node switches.

If all 4096 ports are used the filesystem will share uplink bandwidth with the nodes attached to its node switch.

Page 36: Quadrics, Inc. June 2006 – Confidential slide 1 QsNet III and QsTenG Networks for High Performance Computing 12 th June 2006.

Quadrics, Inc. June 2006 – Confidential slide 36

Quadrics Products Compared

QsNetI I QsTenG QsNetI I I

ProtocolQuadrics

proprietary10GB Ethernet

Quadrics proprietary

Physical layer 10 bits @ 1.333GHz

CX4 4 bits @ 3.125Gb/s

DDR CX4 4 bits @ 6.25Gb/s

Unidirectional rate 920 Mbytes/s 1000 Mbytes/s 2000 Mbytes/s

Bidirectional rate 900 Mbytes/s 1000 Mbytes/s 4000 Mbytes/s

Switch arity 8 12 16/32

Switch hop latency 20ns 450ns 40ns

NIC Elan4 Commodity or

Elan5-10G RDMA Elan5 - QsNetIII

Bus Interface PCI-XPCI-X or PCI

Express16x PCI Express

MPI Latency 1.3uS ~10uS ~1.3uS