Top Banner
The Great White SHARCNET
43

The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Dec 25, 2015

Download

Documents

Edgar Parsons
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

The Great White SHARCNET

Page 2: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

SHARCNET:

Building an Environment to Foster Computational Science

Shared Hierarchical Academic Research Computing Network

Page 3: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

How do you study the first milliseconds of the universe?

And then change the rules?

Page 4: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

How do you study materials that can’t be made in a lab (yet)?

On the surface of the sun? At the centre of the earth?

Page 5: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

How do you study the effects of an amputated limb on blood flow through the heart?

How do you repeat the experiment? Where do you get volunteers?

Page 6: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Increasingly, the answer to “how” in these questions is by using a computer

In addition to experimental and theoretical, we now have computational science!

Page 7: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

So what does this have to do with Western and SHARCNET????

And GreatWhite?

Page 8: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

SHARCNET was created to meet the needs of computational researchers in South Western Ontario.

Western is the lead institution and the administrative home of SHARCNET.

Page 9: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Vision

To establish a world-leading, multi-university and college, interdisciplinary institute with an active academic-industry partnership, enabling forefront computational research in critical areas of science, engineering and business.

Page 10: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Focus on Infrastructure and Support

Support development of computational approaches for research in science, engineering, business, social sciences

Computational facilities People

Page 11: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Computational Focus

Provide world-class computational facilities and support

Explore new computational models• Build on “Beowulf” model

Page 12: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

“Beowulf”

6th century Scandinavian hero Yes, but not in this context:

Collection of separate computers connected by standard communications

What’s so interesting about this?

Page 13: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

First Beowulf

First explored in 1994, two researchers at NASA

Built out of “common” computers• 16 Intel ‘486 processors• 10Mb ethernet

To meet specific computational needs

Page 14: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Beowulf “Philosophy”

Build out of “off the shelf computational components”

Take advantage of increased capabilities, reliability and cost effectiveness of mass market computers

Take advantage of parallel computation “Price/Performance”: “cheap” supercomputing!

Page 15: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Growth of Beowulf

Growth in number and size of “Beowulf clusters” Continued development of mass produced

computational elements Continued development of communication

technologies Development of new parallel programming

techniques

Page 16: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

SHARCNET

Sought to exploit “Beowulf” approach High performance clusters: “Beowulf on steroids”

• Powerful “off the shelf” computational elements• Advanced communications

Geographical separation (local use) Connect clusters: emerging optical

communications

Page 17: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Great White! Processors:

• 4 alpha processors: 833Mhz (4p-SMP)• 4 Gb of memory• 38 SMPs: a total of 152 processors

Communications• 1 Gb/sec ethernet• 1.6 Gb/sec quadrics connection

November 2001: #183 in the world• Fastest academic computer in Canada• 6th fastest academic computer in North America

Page 18: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Great White

Page 19: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

SHARCNET

Extend “Beowulf” approach to clusters of high performance clusters

Connect clusters: “clusters of clusters”• Build on emerging optical communications• Initial configuration used optical equipment

from telecommunications industry Collectively a supercomputer!

Page 20: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Clusters Across Universities

GUELPH

MACUWO

108

128

48

152

Optical communication

Page 21: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Experimental Computational Environment

Deeppurple

48 processors

Optical communication (8Gb/sec)

Greatwhite

152 processors

Page 22: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Technical Issues Resource Management

• How to intelligently allocate and monitor resources• Availability

– Failure rates multiplied by number of systems– Job migration, check pointing

• Performance User Management

Page 23: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Technical Issues Data Management

• Must get the data to the processors

• Some data sets are too large to move

• Some HPC centers now focusing on “Data Grids” vs “Computation Grids”

Most Sharcnet programs use MPI which can run either over TCP or the Quadrics transport layer.

Page 24: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

SHARCNET: More than this!

Not just computational facilities Focus on computational resources to provide

innovative science• Support• Build research community

CFI-OIT: Infrastructure ORDCF: People and programs

Page 25: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Objectives

Provide state of the art computational facilities Develop a network of HPC Clusters Facilitate & enable world class computational research Increase pool of people skilled in HPC techniques & processes Evaluate & create computational Grid as a means of providing

supercomputing capabilities Achieve log term self sustainability Create major business opportunities in Ontario

Page 26: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Operating Principles

Partnership among institutions Shared resources Equality of opportunity Services at no cost to researchers

Page 27: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Partners

Academic: Industry:

University of Guelph Hewlett PackardMcMaster University Quadrics Supercomputing WorldThe University of Western Ontario Platform ComputingUniversity of Windsor Nortel NetworksWilfrid Laurier University Bell CanadaSheridan CollegeFanshawe College

Page 28: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Support Programs: Part 1

Chairs Program: up to 14 new faculty Fellowships: approx. $1 Million per year

• Undergraduate summer jobs• Graduate scholarships• Post doctoral fellowships

Page 29: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Support Programs: Part 2

Technical Staff• System administrators at sites• HPC consultants at sites

Workshops Conferences

Page 30: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Results??

Researchers from a variety of disciplines• Chemistry• Physics• Economics• Biology

Beginning to “ramp up”

Page 31: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Chemistry

Model chemical systems and processes at the atomic and electronic levels• New quantum chemical techniques• New computational methods

For example: Molecular dynamics simulation of hydrogen formation in a single-site Olefin polymerization catalyst

Page 32: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Economics Research on factors that influence people to

retire Model incorporates both health and financial

factors• Previous models looked at one or other• Much more complex: difficult to estimate

parameters

Page 33: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Materials Understand friction and

lubrication at molecular and atomic levels

Friction between polymer bearing surfaces

Two polymer bearing surfaces in sliding motion and good solvent conditions

Green: upper wall Red: lower wall

Page 34: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Astrophysics

Galaxy merger Approximately 100,000

particles (stars and dark matter)

Page 35: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Astrophysics

Forming giant planets from protoplanetary disks

Shows evolution of disk over about 200 years

Page 36: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Materials: Granular matter:

Understand flow of granular matter

Study the effectiveness of mixing

Study the effect of different components on mixing

Page 37: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

The Future?

New members: Southwestern Ontario Support for new science

• Greater computation• Storage facilities

New areas• Bioinformatics

Page 38: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

The Future? New Partners (Waterloo, Brock, York, UOIT) Additional Capacity

– Storage• ½ petabyte across 4 sites (multistage performance)

– Network• 10 Gb/s core (UWO, Waterloo, Guelph, Mac)• 1 Gb/s to other SHARCNET sites• 10 Gb/s to Michnet, HPCVL (?)

– Upgrades• Large capability machines at Mac, UWO, Guelph• Large capacity machines at Waterloo• Increased development sites

Page 39: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

The Future? Additional Capabilities

–Visualization Total investment

–~$49M (actually there is an additional 7 million cash from HP).

2004-2005–With the new capabilities, Sharcnet could be in

the top 100 or 150 of supercomputers.–Will be the fastest supercomputer of its kind –

I.e.,a distributed system where nodes are clusters.

Page 40: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.
Page 41: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.
Page 42: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

Lake OntarioLakeHuron

I

I

X

Iα α

I

S

S

Windsor

Western

Waterloo

Guelph

UOIT

YorkFields

Brock

McMaster

Laurier

Sheridan

Fanshawe

Robarts

Perimeter

Lake Erie

T

αG

I

I

10 Gb/sα Redeployed Alphas

1 Gb/s

I Itanium clusterX Xeon clusterS SMPG Grid LabT Interconnect Topology Cluster

Tape

EVA Disc

MSA Disc

100km

50 mi

(0.1ms)

Page 43: The Great White SHARCNET. SHARCNET: Building an Environment to Foster Computational Science Shared Hierarchical Academic Research Computing Network.

The Future: SHARCNET!