Top Banner
IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003
16

IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

Dec 13, 2015

Download

Documents

Polly Parks
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

IDE disk servers at CERN

Helge Meinhard / CERN-IT

CERN OpenLab workshop

17 March 2003

Page 2: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

Introduction

HEP computing in the past: mostly reading from, processing, and writing to (tape) files sequentially

Mainframe era (until ~ 1995): single machine, CPUs, tape drives, little disk space

In response to scaling problem, development of SHIFT architecture (early 1990s) Scalable farm out of ‘commodity’ components

RISC CPUs (PowerPC, MIPS, Alpha, PA-RISC, Sparc) SCSI disks

Page 3: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

SHIFT architecture

Network- FDDI- Hippi- Myrinet- Ethernet

Tapeserver

Diskserver

Batchserver

Interactiveserver

BatchanddiskSMPNetwork

- Ethernet

Page 4: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

PC batch nodes

1995: First studies at CERN of PCs as batch nodes (Windows NT)

1997 onwards: Rapidly growing interest in Linux (on IA32 only)

1998/99: First production farms for interactive and batch services running Linux on PC hardware at CERN

Page 5: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

PC disk servers

1997/98: Prototypes with SCSI disks 1998/99: Prototypes with EIDE disks

Different IDE adapters Not RAIDed

1999/2000: First Jumbo servers (20 x 75 GB) put into production

2001: First rack-mounted systems 2002: 97 new servers (54 TB usable) 2003: 1.3 TB usable in one server at 13 kCHF Total usable capacity today: ~ 200 TB

Page 6: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

1997: ~700 GB SCSI/Sparc

Page 7: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

2000/2001: 750 GB PC/EIDE (1)

Page 8: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

2000/2001: 750 GB PC/EIDE (2)

Page 9: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

2002: 670 GB PC/EIDE

2 systems

Page 10: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

Assumed price per GB usable

0

50

100

150

200

250

300

350

400

450

1994 1995 1996 1997 1998 1999 2000 2001 2002

Time

CH

F /

GB

Disks only

Complete systems

SCSI/RISC

EIDE/PC

Page 11: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

Real price per GB incl. infrastructure

0

5

10

15

20

25

30

35

40

45

50

Oct-

00

Nov-0

0

Dec-0

0

Jan-

01

Feb-0

1

Mar

-01

Apr-0

1

May

-01

Jun-

01

Jul-0

1

Aug-0

1

Sep-0

1

Oct-

01

Nov-0

1

Dec-0

1

Jan-

02

Feb-0

2

Mar

-02

Apr-0

2

May

-02

Jun-

02

Jul-0

2

Aug-0

2

Sep-0

2

Oct-

02

Nov-0

2

Dec-0

2

Jan-

03

Feb-0

3

Date

CH

F /

GB

Page 12: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

Capacity per server

0

500

1000

1500

2000

2500

3000

3500

Oct-

00

Nov-0

0

Dec-0

0

Jan-

01

Feb-0

1

Mar

-01

Apr-0

1

May

-01

Jun-

01

Jul-0

1

Aug-0

1

Sep-0

1

Oct-

01

Nov-0

1

Dec-0

1

Jan-

02

Feb-0

2

Mar

-02

Apr-0

2

May

-02

Jun-

02

Jul-0

2

Aug-0

2

Sep-0

2

Oct-

02

Nov-0

2

Dec-0

2

Jan-

03

Feb-0

3

GB

Gross

Usable

Page 13: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

Today’s servers: Specifications

19” rackmount, IA32 processor(s), 1 GB, 2x80 GB system disks, GigE (1000BaseT), redundant power supplies

>500 GB usable space on data disks Hardware RAID offering redundancy Hot-swap disk trays

Performance requirements network – disk: 50 MB/s reading from server @ 500 GB 40 MB/s writing to server @ 500 GB

5 years on-site warranty

Page 14: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

Lessons learnt

Capacity is not everything, for good performance need CPU, memory, RAID cards Good OS and application software Network connectivity Large number of spindles

Firmware of RAID controllers and disks critical Redundancy (RAID) is a must, required

performance possible only with mirroring (RAID 1) so far

Page 15: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

Outlook

Good price/performance has risen interest in other application domains at CERN AFS and MS DFS servers Web servers, mail servers Software servers (Linux installation) Data base servers (Oracle, Objectivity/DB)

Access pattern of physics analysis likely to change Investigating different file systems (XFS), RAID 5 (in

software), … Architecture constantly being reviewed

Alternatives investigated: data disks scattered over large number of batch nodes; SAN

Page 16: IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.

Conclusion

Architecture of early 1990s still valid May even carry us into LHC era…

Important improvements made Price/performance Reliability (RAID)

Will review architecture soon (2003) New application areas New access patterns for physics analysis