Presentation of the 16th List

1

Presentation of the 16th List

Hans Werner Meuer, University of Mannheim Erich Strohmaier, University of TennesseeJack J. Dongarra, University of TennesseHorst D. Simon, NERSC/LBNL

SC2000, Dallas, TX, November 4-10, 2000 BoF Session, November 8, 5:30pm

TOP500 BoFTOP500 BoF

• Presentation of the 16th Top500 ListPresentation of the 16th Top500 List• TOP500 Background Information

and general Market Overview.• Geographical Distribution• Manufacturers• Performance Development• Architectures, Technology• Future TOP500 Ideas and Plans

2

TOP500 Motivation

• Basis for analysing the HPC market

• Quantification of observations

• Detection of trends

– Market

– Architecture

– Technology

TOP500TOP500- Listing of the 500 most powerfulComputers in the World

- Yardstick: Rmax from LINPACK MPPAx=b, dense problem

- Updated twice a yearSC‘xy in the States in NovemberMeeting in Mannheim, Germany in June

- All data available from www.top500.org

8N_J

7FYJ

955�UJWKTWRFSHJ

3

,Q��D�FRPSXWDWLRQ�WKDW�WRRN��IXOO�\HDU�WR�FRPSOHWHFDQ�QRZ�EH�GRQH�LQ��PRQWK�

Fastest Computer Over Time

TMCCM-2(2048)

Fujitsu VP-2600

Cray Y-MP (8)

05

101520253035404550

1990 1992 1994 1996 1998 2000

Year

GF

lop/

s

,Q��D�FRPSXWDWLRQ�WKDW�WRRN��IXOO�\HDU�WR�FRPSOHWHFDQ�QRZ�EH�GRQH�LQ��GD\V�

Fastest Computer Over Time

HitachiCP-PACS

(2040)IntelParagon

(6788)

FujitsuVPP-500

(140)TMC CM-5(1024)

NEC SX-3(4)

TMCCM-2(2048)

Fujitsu VP-2600

Cray Y-MP (8)

050

100150200250300350400450500

1990 1992 1994 1996 1998 2000

Year

GF

lop/

s

4

,Q��D�FRPSXWDWLRQ�WKDW�WRRN��IXOO�\HDU�WR�FRPSOHWHFDQ�WRGD\�EH�GRQH�LQ��KRXU�

Fastest Computer Over TimeASCI White

Pacific(7424)

ASCI BluePacific SST

(5808)

SGI ASCIBlue

Mountain(5040)

Intel ASCI Red

(9152)Hitachi

CP-PACS(2040)

IntelParagon(6788)

FujitsuVPP-500

(140)

TMC CM-5(1024)

NEC SX-3

(4)

TMCCM-2(2048)

Fujitsu VP-2600

Cray Y-MP (8)

Intel ASCIRed Xeon

(9632)

0500

100015002000250030003500400045005000

1990 1992 1994 1996 1998 2000

Year

GFl

op/s

- Author since 6/1993 and “11/2000”

Jack Dongarra Horst Simon

TOP500 AuthorsTOP500 Authors- Started in spring 1993 by:

Hans W. Meuer and Erich Strohmaier

5

TOP500 Status

• 1. List in June 1993

• Accepted by users and manufacturers

• 16. List on November 2, 2000

6

TOP500 list TOP500 list -- Data shownData shown• Manufacturer Manufacturer or vendor

• Computer Type indicated by manufacturer or vendor

• Installation Site Customer

• Location Location and country

• Year Year of installation/last major update

• Customer Segment Academic,Research,Industry,Vendor,Class.

• # Processors Number of processors

• Rmax Maxmimal LINPACK performance achieved

• Rpeak Theoretical peak performance

• Nmax Problemsize for achieving Rmax

• N1/2 Problemsize for achieving half of Rmax

� 1ZRUOG�� 3RVLWLRQ�ZLWKLQ�WKH�723��UDQNLQJ

TOP10Rank Manufacturer Computer

Rmax

[TF/s]Installation Site Country Year Area of

Installation# Proc

1 IBM ASCI WhiteSP Power3

4.93 Lawrence LivermoreNational Laboratory

USA 2000 Research 8192

2 Intel ASCI Red 2.38 Sandia National Laboratory USA 1999 Research 9632

3 IBM ASCI Blue PacificSST, IBM SP

604E



4 SGI ASCI BlueMountain

1.61 Los AlamosNational Laboratory


5 IBM SP Power3375Mhz

1.42 IBM/Naval OceanographicOffice (NAVOCEANO)


6 IBMSP Power3

375Mhz1.18

National Centers forEnvironmental Predicition


7 Hitachi SR8000-F1 1.04 Leibniz Rechenzentrum Germany 2000 Academic 112

8 IBM SP Power3375MHz 8way

0.93 San Diego SupercomputerCenter

USA 2000 Academic 1152

9 Hitachi SR8000-F1 0.92 High Energy AcceleratorResearch Organization/ KEK

Japan 2000 Research 100

10 Cray Inc. T3E 1200 0.89 Government USA 1998 Classified 1084

7

TOP10Rank Manufacturer Computer

Rmax

[TF/s]Installation Site Country Year Area of

Installation# Proc

1 IBM ASCI WhiteSP Power3



2 Intel ASCI Red 2.38 Sandia National Laboratory USA 1999 Research 9632

3 IBM ASCI Blue PacificSST, IBM SP

604E



4 SGI ASCI BlueMountain

1.61 Los AlamosNational Laboratory


5 IBM SP Power3375Mhz

1.42 IBM/Naval OceanographicOffice (NAVOCEANO)


6 IBMSP Power3

375Mhz1.18

National Centers forEnvironmental Predicition


7 Hitachi SR8000-F1 1.04 Leibniz Rechenzentrum Germany 2000 Academic 112

8 IBM SP Power3375MHz 8way

0.93 San Diego SupercomputerCenter

USA 2000 Academic 1152

9 Hitachi SR8000-F1 0.92 High Energy AcceleratorResearch Organization/ KEK

Japan 2000 Research 100

10 Cray Inc. T3E 1200 0.89 Government USA 1998 Classified 1084

Architectures

Single Processor

SMP

MPP

SIMD Constellation Cluster - NOW

0

100

200

300

400

500

Jun-9

3

Nov-93

Jun-9

4

Nov-94

Jun-9

5

Nov-9

5

Jun-9

6

Nov-96

Jun-9

7

Nov-97

Jun-9

8

Nov-9

8

Jun-9

9

Nov-99

Jun-0

0

Nov-00

CluMPs

Y-MP C90

Sun HPC

Paragon

CM5T3D

T3E

SP2

Cluster ofSun HPC

ASCI Red

CM2

VP500

SX3

8

Excerpt from TOP500

Rank Manufacturer Computer Rmax [GF/s]

Installation Site Country Year Area of Installation

# Proc

… … … … … … … …

80 IBM LosLobos 237 University of New Mexico USA 2000 Academic 512

… … … … … … … … …

84 Self-made Cplant Cluster 232 Sandia Nationla Lab USA 1999 Research 580

… … … … … … … … …

96 HPTi ACL-276 196 NOAA USA 1999 Research 276

… … … … … … … … …

126 Self-made CLIC PIII 143 Technical University of

Chemnitz Germany 2000 Academic 528

… … … … … … … … …

413 Self-made NT SuperCluster

63 NCSA USA 1999 Academic 256

Performance Development

88.0 TF/s

1.167 TF/s

59.7 GF/s

4.94 TF/s

0.4 GF/s

55.1 GF/s

Jun-

93

Nov-93

Jun-

94

Nov-94

Jun-9

5

Nov-9

5

Jun-9

6

Nov-9

6

Jun-9

7

Nov-9

7

Jun-9

8

Nov-98

Jun-

99

Nov-99

Jun-

00

Nov-00

Intel XP/S140Sandia

Fujitsu’NWT’ NAL

SNI VP200EXUni Dresden

Hitachi/TsukubaCP-PACS/2048

IntelASCI Red

Sandia

IBMASCI White

LLNLN=1

N=500

SUM

IBM SP PC604e130 processors

Alcatel

1 Gflop/s

1 Tflop/s

100 Mflop/s

100 Gflop/s

100 Tflop/s

10 Gflop/s

10 Tflop/s

9

Performance Development

ASCI

Earth Simulator

Jun-9

3

Jun-9

4

Jun-9

5

Jun-9

6

Jun-9

7

Jun-9

8

Jun-9

9

Jun-0

0

Jun-0

1

Jun-0

2

Jun-0

3

Jun-0

4

Jun-0

5

N=1

N=500

Sum

1 GFlop/s

1 TFlop/s

1 PFlop/s

100 MFlop/s

100 GFlop/s

100 TFlop/s

10 GFlop/s

10 TFlop/s

Continents

USA/Canada

Europe

Japan

others

0

100

200

300

400

500

Jun-

93

Nov-9

3

Jun-9

4

Nov-9

4

Jun-

95

Nov-9

5

Jun-

96

Nov-9

6

Jun-9

7

Nov-97

Jun-

98

Nov-9

8

Jun-

99

Nov-9

9

Jun-

00

Nov-00

10

Continents - Performance

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Jun-

93

Nov-9

3

Jun-9

4

Nov-94

Jun-9

5

Nov-9

5

Jun-9

6

Nov-96

Jun-

97

Nov-97

Jun-9

8

Nov-98

Jun-

99

Nov-9

9

Jun-

00

Nov-00

Per

form

ance

USA/Canada

other

Japan

Europe

Europe - Countries

Germany

UK

France Scandinavia

BeneluxSwitzerland

others

0

50

100

150

Jun-

93

Nov-9

3

Jun-

94

Nov-9

4

Jun-

95

Nov-9

5

Jun-

96

Nov-9

6

Jun-

97

Nov-9

7

Jun-

98

Nov-9

8

Jun-

99

Nov-9

9

Jun-

00

Nov-0

0

11

Producers

USA

JapanEurope

0

100

200

300

400

500

Jun-

93

Nov-9

3

Jun-

94

Nov-9

4

Jun-

95

Nov-9

5

Jun-

96

Nov-9

6

Jun-

97

Nov-9

7

Jun-

98

Nov-9

8

Jun-

99

Nov-9

9

Jun-

00

Nov-0

0

Producers - Performance

0%10%20%30%40%50%60%70%80%90%

100%

Jun-

93

Nov-9

3

Jun-

94

Nov-9

4

Jun-

95

Nov-9

5

Jun-

96

Nov-9

6

Jun-

97

Nov-9

7

Jun-

98

Nov-9

8

Jun-

99

Nov-9

9

Jun-

00

Nov-0

0

Perf

orm

an

ce

USA

Japan

Europe

12

Manufacturer

Cray

SGI

IBM

Sun

Convex/HP

TMC

Intel

FujitsuNEC

Hitachiothers

0

100

200

300

400

500

Jun-9

3

Nov-93

Jun-9

4

Nov-94

Jun-9

5

Nov-95

Jun-9

6

Nov-96

Jun-9

7

Nov-97

Jun-9

8

Nov-98

Jun-9

9

Nov-99

Jun-0

0

Nov-00

Manufacturer

Cray

SGI

IBM

SunConvex/HP

TMC

Intel

Fujitsu

NEC

Hitachiothers

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Jun-9

3

Nov-93

Jun-9

4

Nov-94

Jun-9

5

Nov-95

Jun-9

6

Nov-96

Jun-9

7

Nov-97

Jun-9

8

Nov-98

Jun-9

9

Nov-99

Jun-0

0

Per

form

ance

13

www.top500.org

Kflops per Inhabitant

183

111 10190 89 74

49 29

509

0

100

200

300

400

500

600

US

A

Sca

ndin

avia

Ger

man

y

UK

Japa

n

Sw

itzer

land

Fran

ce

New

Zea

land

Luxe

mbo

urg

14

Customer TypeVendor

Research

Industry

Academic

Classified

0

100

200

300

400

500

Jun-

93

Nov-9

3

Jun-

94

Nov-9

4

Jun-

95

Nov-9

5

Jun-

96

Nov-9

6

Jun-

97

Nov-9

7

Jun-

98

Nov-9

8

Jun-

99

Nov-9

9

Jun-

00

Nov-0

0

Customer Types - Performance

Research

Industry

Academic

ClassifiedVendor

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Jun-

93

Nov-9

3

Jun-

94

Nov-9

4

Jun-

95

Nov-9

5

Jun-

96

Nov-9

6

Jun-

97

Nov-9

7

Jun-

98

Nov-9

8

Jun-

99

Nov-9

9

Jun-

00

Nov-0

0

Pe

rfo

rma

nc

e

15

Industrial Customer Segments

Engineering

Commercial

Unknown

0

50

100

150

200

250

Jun-9

3

Nov-93

Jun-9

4

Nov-94

Jun-9

5

Nov-95

Jun-9

6

Nov-96

Jun-9

7

Nov-97

Jun-9

8

Nov-98

Jun-9

9

Nov-99

Jun-0

0

Nov-00

Processor Type

Scalar

Vektor

SIMD

0

100

200

300

400

500

Jun-9

3

Nov-93

Jun-9

4

Nov-94

Jun-9

5

Nov-9

5

Jun-9

6

Nov-96

Jun-9

7

Nov-97

Jun-9

8

Nov-9

8

Jun-9

9

Nov-99

Jun-0

0

Nov-00

16

Chip Technology

CMOS/off the shelf

CMOS/ proprietary

ECL

0

100

200

300

400

500

Jun-

93

Nov-9

3

Jun-9

4

Nov-9

4

Jun-9

5

Nov-9

5

Jun-

96

Nov-9

6

Jun-

97

Nov-9

7

Jun-9

8

Nov-98

Jun-

99

Nov-9

9

Jun-

00

nov00

0

Chip Technology

Alpha

PowerHP

intel

MIPS

SUNother COTS

proprietary

0

100

200

300

400

500

Jun-9

3

Nov-9

3

Jun-9

4

Nov-9

4

Jun-9

5

Nov-9

5

Jun-9

6

Nov-9

6

Jun-9

7

Nov-9

7

Jun-9

8

Nov-9

8

Jun-9

9

Nov-9

9

Jun-0

0

Nov-0

0

17

TOP Centers

• Two approaches (No ‘classified’ centers):

• Sum up all TOP500 systems for each site– Snapshot only

• “Try” on historical perspective:– Sum up all systems over all lists all lists * 0.5 years* 0.5 years

= Total deliverable Linpack flop years

– This give a ‘discounted’ credit for history

TOP10 Centers - “snap-shot”Rank Site

∑ Rmax

[TF/S]Current Best Country

1 Lawrence Livermore National Laboratory 7.7 1, 3, 32, 36 USA

2 Sandia National Laboratory 2.38 2, 408 USA

3 Los Alamos National Laboratory 2.36 4, 19, 401 USA

4 Naval Oceanographic Office(NAVOCEANO)

2.17 5, 21, 326 USA

5 US Army Research Laboratory (ARL) 1.32 28, 138, 253-258, 424, 425

USA

6 Charles Schwab 1.28 15, 34 USA

7 ECMWF 1.20 12, 90, 214 UK

8 National Center for EnvironmentalPrediction

1.18 9 USA

9 Leibniz Rechenzentrum 1.14 11, 196 Germany

10 University of Tokyo 1.11 13, 85 Japan

18

TOP10 Centers - “integrated”

Rank Site∑ t*Rmax

[TFYears]Current Best Country

1 Sandia National Laboratory 8.0 2, 408 USA

2 Lawrence Livermore National Laboratory 7.5 1, 3, 32, 36 USA

3 Los Alamos National Laboratory 6.4 4, 19, 401 USA

4 University of Tokyo 3.3 13, 85 Japan

5 Naval Oceanographic Office(NAVOCEANO)

2.9 5, 21, 326 USA

6 United Kingdom Meteorological Office 2.7 26, 29 UK

7 Forschungszentrum Juelich (FZJ) 2.1 40, 83 Germany

8 NERSC/LBNL 2.0 42, 60 USA

9 Oak Ridge National Laboratory 1.9 17, 70, 402 USA

10 ECMWF 1.8 12, 90, 214 UK

Performance Distribution

0

1

2

3

4

5

1 10 100 1000Rank

Per

form

ance

[TF

lop/

s]

19

Cumulative Performance

0102030405060708090

100

1 10 100 1000Rank

Pe

rfo

rma

nc

e [

TF

lop

s]

88 TF/s

Rank 1/2

Performance Distribution

Rank of 1/2 TOP500 Performance

0

10

20

30

40

50

60

70

Jun-9

3

Nov-93

Jun-9

4

Nov-94

Jun-9

5

Nov-95

Jun-9

6

Nov-96

Jun-9

7

Nov-97

Jun-9

8

Nov-98

Jun-9

9

Nov-99

Jun-0

0

Nov-00

20

What’s new with the TOP500?

•• HPLHPL: High Performance LinpackAntoine Petitet and Clint Whaley, ICL, UTK

– icl.cs.utk.edu/hpl

– Needs only• MPI

• BLAS or VSIPL

– Highly scalable and efficient for the whole range of system sizes we see

What’s new with the TOP500?

• New database interface at www.top500.org– More categories and variables to select or list

– experimental cross-linked lists

• Ad’s corner(Ad Emmen)

• Architecture descriptions(Aad van der Steen)

21

What’s next with the TOP500?

• What’s now?

• New application areas of ‘Supercomputers’

• New architectures for ‘Supercomputers’

• ‘Self-made’ systems

Implications

• Commercial DB systems– “don’t care about floating point”

• New architectures - ‘compute farms’– “not designed to run Linpack”

• “Self-made” cluster– no vendor to measure Linpack

– every system is different

22

TOP500 Plans

• Talks with experts since 3 years on how to ‘resolve’ these issues

• Q: “How can we provide better information for different application and architecture domains?”

• A: “Make additional lists”

TOP500 Plans

• Keep current TOP500– Continuity, trends, 16 existing editions

– 17th: Supercomputer’2001Supercomputer’2001Mannheim/Heidelberg, Germany, June 21-23, 2001

• The current TOP500 will remain ‘center’ of future activities

23

TOP500 Plans

• Assemble additional lists for special purposes - first try:– “TOP100 (commodity) cluster”

– Jointly with TFCC

– But what is a cluster ???

– Which benchmark?• Initially without

TOP500 Plans

• Collaborate with research centers (NERSC, ICL, ...) which develop new benchmarks

• Feedback about benchmarks

• TOP500.org will provide – free access to benchmark results and

– possibilities for ‘re-ranking’

24

TOP500 Future

• Eventually:

• Hierarchical structure– Master List (current TOP500?)

– Domain specific sub-lists ranked differently

• Historical information about systems

• Access to benchmark and architecture information

Presentation of the 16th List

Documents