Conference xxx - August 2003 Sverker Holmgren SNIC Director SweGrid A National Grid Initiative within SNIC Swedish National Infrastructure for Computing.

Conference xxx - August 2003

Sverker HolmgrenSNIC Director

SweGridA National Grid Initiative

within

SNICSwedish National Infrastructure for Computing

NORDUnet 2008-04-10

SNIC?

A metacenter, organized within Vetenskapsrådet. Formed 2003, but builds on much older computer centers.

• Mission: • Provide funding for computing resources for academic

research in Sweden• Coordinate investments, user support and competence• Provide mechanisms for resource allocation• Fund and coordinate development projects• Manage the Swedish interface to large international

collaborations and projects• Host the Swedish National Graduate School in

Scientific Computing (NGSSC)

NORDUnet 2008-04-10

SNIC?

• Means:• Work by the 6 SNIC centers • A board and a very small executive organization• Strategic plan; “The Swedish Computing Landscape”

Current version: 2006-2009 New version being produced right now: 2008-2011

• HPC2N (Umeå)• UPPMAX (Uppsala)• PDC (Stockholm)• NSC (Linköping)• C3SE (Göteborg)• LUNARC (Lund)

NORDUnet 2008-04-10

SNIC 2008

Funding: Approx. 9,2 MEUR (significant increase from earlier)

HPC resources Grid resources Storage resources

Allocation procedures User support

Outreach

Development projects (HPC, Grid, Storage)

Community building

Participation in international collaboration (PRACE-DS, EGI-DS, NDGF (Nordic T1), EGEE, ...)

Education (NGSSC)Proposals and reports(e.g. SweScience)

NORDUnet 2008-04-10

SNIC 2008• Computational resources:

• Large-scale national resources• Group-specific large-scale resources (KAW

collaboration)• Foundation level resources (at all centers)• Currently: 22 systems, 13490 cores• Largest: Neolith at NSC, 6440 cores, 60 Tflop

• Currently about 400 user groups, growing• Significant new investments for SNIC users now

• About 20000 new cores in 8 months

• SweGrid is an integrated part of SNIC• Goal: All SNIC systems available via Grid interface

NORDUnet 2008-04-10

SweGrid?

• The Swedish NGI (MoU signed between SNIC and centers)

• The Swedish production grid system• Collaboration SNIC – the Swedish LHC consortium (LHCK)

NORDUnet 2008-04-10

SweGrid Phase I – A Production Testbed

• Initiative from • Users

High energy physics Life Science Earth Sciences Space & Astro Physics

• All HPC-centers in Sweden• IT-researchers wanting to research Grid technology

• First generation hardware in use spring 2004• 6 clusters with 100 processors each• 410 TB storage

NORDUnet 2008-04-10

SweGrid Production Testbed

• Total budget 3.6 MEuro• 6 GRID nodes• 600 CPUs

• IA-32, 1 processor/server • 2.8 GHz Intel P4 • 2 Gbyte • Gigabit Ethernet

• 12 TByte temporary storage • FibreChannel for bandwidth

• 410 TByte nearline storage• 140 TByte disk• 270 TByte tape

• 1 Gigabit direct connection to SUNET (10 Gbps)

NORDUnet 2008-04-10

SweGrid status 2007• Nodes installed January 2004, now becoming old

• Extensive use of the resources• Resource allocation via the national allocations committee (SNAC)• GRID queues through the NorduGrid middlware – ARC• Some nodes also available using gLite. Part of EGEE North

Federation resources• 17 non-LHCK groups applied for SweGrid resources• About 60 national users

• 1/3 of SweGrid is dedicated to HEP (200 CPUs)

• Contributing to LCG challenges

• Forms the core of the Northern EGEE ROC

• Accounting has been introduced – SGAS

NORDUnet 2008-04-10

National projects and services 2007

• Grid resource brokering, Umeå

• Grid portals (general, application oriented), Umeå, Lund

• Grid accounting, SweGrid Accounting System (SGAS), Stockholm, Uppsala, Umeå

• Grid security research – focused on trust models, Stockholm, Umeå and Uppsala

• Parallel Object Query System for Expensive Computations (POQSEC) on Grids, Uppsala

• National Helpdesk, all SweGrid centers

• National storage solutions, all SNIC centers

NORDUnet 2008-04-10

Persistent storage on SweGrid?

Size Administration

BandwidthAvailability

1

2

3This was the choice!

NORDUnet 2008-04-10

The first users of SweGrid

0

20000

40000

60000

80000

100000

120000

140000

HEP

Physic

s

Bio S

cien

ce

Chemis

try

Enginee

ring

Earth

Sci

ence

s

Research Area

ho

urs

/mo

nth

NORDUnet 2008-04-10

User survey 2005:How have you found porting your applications to SweGrid to be?

0

2

4

6

8

10

12

14

16

18

20

Users

Very difficult

Difficult

Cumbersome

Easy

Very easy

NORDUnet 2008-04-10

User survey 2005: What is your overall impression of use of SweGrid resources?

0

2

4

6

8

10

12

14

16

18U

sers

Very Good

Good

Fair

Poor

NORDUnet 2008-04-10

Summarizing the SweGrid Production Testbed effort

• Unified the Swedish Supercomputing Community

• Raised additional private funding for computing infrastructure

• Spawned national Grid R&D projects

• Created early user Grid awareness

• Created an interface to EU-projects• EGEE

Regional operations center co-operated with SweGrid Co-ordination of EGEE security

• Co-ordination of Baltic Grid Initiative

NORDUnet 2008-04-10

SweGrid Phase II

• New clusters and storage are being installed• Almost 3000 cores

LHCK (33%) and general use (66%)

• 340 TB disk Mainly for LHCK

• Long-term plan for further upgrades• During 2008: Additional 320 cores, 750 TB disk, 3

new tape storage facilities

• Other SNIC systems (capability clusters, SMP) also available within SweGrid

NORDUnet 2008-04-10

New projects and services, started

• National storage project – Next phase (“SweStore”)• Partly using SweGrid storage• Collaboration with Database Infrastructure Committee (DISC)• Establishing collaboration with Nordic centers

• Application portal, middleware independent• Chemistry• Genetics• Astrophysics• …

• Bioinformatics workflow system

• BioGrid (NDGF)

• More work on accounting, scheduling etc

NORDUnet 2008-04-10

Examples of proposed projects

• Grid portals/interfaces, “science gateways”

• Cloud computing initiative (“SweCloud”)

• Study on impact of virtualization techniques

• SOA-based middleware-independent tools• Accounting

• Job management framework

• Scheduling

NORDUnet 2008-04-10

Swedish Grid Strategy

• Described in the SNIC Landscape Document• SweGrid/NGI is an integrated part of SNIC• SweGrid is an essential part of the infrastructure for

Swedish e-Science• SNIC proposal for a Swedish e-Science program• Coordination of SweGrid and HPC, e.g “PRACE-Grid”• Swedish participation in EGI• Swedish participation in e.g. ESFRI projects

• All SNIC resources should be available via SweGrid• Alternative to traditional means of access• New applications enabled• New interfaces

• R&D efforts to provide services and components• SweGrid provides a basis for Swedish participation in

major international projects

NORDUnet 2008-04-10