PARAID: A Gear-Shifting Power-Aware RAID

1

PARAID: A PARAID: A Gear-ShiftingGear-ShiftingPower-Aware RAIDPower-Aware RAID

Charles Weddle, Mathew Oldham, Jin Qian, An-I Andy Wang – Florida St. UniversityCharles Weddle, Mathew Oldham, Jin Qian, An-I Andy Wang – Florida St. University

Peter Reiher – University of California, Los AngelesPeter Reiher – University of California, Los Angeles

Geoff Kuenning – Harvey Mudd CollegeGeoff Kuenning – Harvey Mudd College

2

MotivationMotivation

Energy costs are risingEnergy costs are rising An increasing concern for serversAn increasing concern for servers

No longer limited to laptopsNo longer limited to laptops

Energy consumption of disk drivesEnergy consumption of disk drives 24% of the power usage in web servers24% of the power usage in web servers 27% of electricity cost for data centers27% of electricity cost for data centers More energy More energy more heat more heat more cooling more cooling lower lower

computational density computational density more space more space more costs more costs

Is it possible to reduce energy consumption Is it possible to reduce energy consumption without degrading performance while without degrading performance while maintaining reliability?maintaining reliability?

PARAID: A Gear-Shifting Power-Aware RAID

3

ChallengesChallenges

EnergyEnergy Not enough opportunities to spin down RAIDsNot enough opportunities to spin down RAIDs

PerformancePerformance Essential for peak loadsEssential for peak loads

ReliabilityReliability Server-class drives are not designed for Server-class drives are not designed for

frequent power switchingfrequent power switching


4

Existing WorkExisting Work

Most trade performance for energy Most trade performance for energy savings directly savings directly e.g. vary speed of diskse.g. vary speed of disks

Most are simulated resultsMost are simulated results


5

ObservationsObservations

RAID is configured for peak performanceRAID is configured for peak performance RAID keeps all drives spinning for light loadsRAID keeps all drives spinning for light loads

Unused storage capacityUnused storage capacity Over-provision of storage capacityOver-provision of storage capacity Unused storage can be traded for energy savingsUnused storage can be traded for energy savings

Fluctuating loadFluctuating load Cyclic fluctuation of loadsCyclic fluctuation of loads Infrequent on-off power transitions can be effectiveInfrequent on-off power transitions can be effective


6

Performance vs. Energy Performance vs. Energy OptimizationsOptimizations

Performance benefitsPerformance benefits Realized under heavy loadsRealized under heavy loads

Energy benefitsEnergy benefits Realized instantaneously Realized instantaneously

7

Power-Aware RAIDPower-Aware RAID

Skewed striping for energy savingsSkewed striping for energy savings

Preserving peak performancePreserving peak performance

Maintaining reliabilityMaintaining reliability

EvaluationEvaluation

ConclusionConclusion


8

Skewed StripingSkewed Striping for Energy Saving for Energy Saving

Use over-provisioned spare storageUse over-provisioned spare storage Organized into hierarchical overlapping subsetsOrganized into hierarchical overlapping subsets


RAID

1 2 3 4 5

9


Each set analogous to gears in automobilesEach set analogous to gears in automobiles


RAID

Gears123

1 2 3 4 5

10


Soft states can be reclaimed for spaceSoft states can be reclaimed for space Persist across rebootsPersist across reboots


RAID

SoftStates

Gears123

1 2 3 4 5

11


Operate in gear 1Operate in gear 1 Disks 4 and 5 are powered offDisks 4 and 5 are powered off


RAID

Gears123

1 2 3 4 5

SoftStates

12


Approximate the workloadApproximate the workload Gear shift into most appropriate gearGear shift into most appropriate gear

Minimize the opportunity lost to save powerMinimize the opportunity lost to save power

Energy( PoweredOn Disks )

Workload( Disk Parallelism )

Conventional RAID PARAID

workload


13


Adapt to cyclic fluctuating workloadAdapt to cyclic fluctuating workload Gear shift when gear utilization threshold is metGear shift when gear utilization threshold is met

time

load

utilization threshold

gear shift


14

Preserving Peak PerformancePreserving Peak Performance

Operate in the highest gear Operate in the highest gear When the system demands peak performanceWhen the system demands peak performance Uses the same disk layoutUses the same disk layout

Maximize parallelism within each gearMaximize parallelism within each gear Load is balancedLoad is balanced Uniform striping patternUniform striping pattern

Delay block replication until gear shiftsDelay block replication until gear shifts Capture block writesCapture block writes


15

Maintaining ReliabilityMaintaining Reliability

Reuse existing RAID levels (RAID-5)Reuse existing RAID levels (RAID-5) Also used in various gearsAlso used in various gears

Drives have a limited number of power Drives have a limited number of power cyclescycles Ration number of power cyclesRation number of power cycles


16

Maintaining ReliabilityMaintaining Reliability

Busy disk stay powered on, idle disks stay powered offBusy disk stay powered on, idle disks stay powered off Outside disks are role exchanged with middle disksOutside disks are role exchanged with middle disks

busydisks

powercycleddisks

idledisks

role exchange

Disk 1

Gear 1Gear 2

Gear 3

Disk 2 Disk 3 Disk 4 Disk 5 Disk 6


17

File system

RAID

PARAID block mappingPARAID block mapping

Disk device driver

User space

Linux kernel

Soft RAIDSoft RAID

Reliability managerReliability manager

Load monitorLoad monitor

Gear managerGear manager

Admin toolAdmin tool

Logical Component DesignLogical Component Design


18

Asymmetric Gear-Shifting PoliciesAsymmetric Gear-Shifting Policies

Up-shift (aggressive)Up-shift (aggressive) Moving utilization average + moving standard Moving utilization average + moving standard

deviation > utilization thresholddeviation > utilization threshold

Downshift (conservative)Downshift (conservative) Modified utilization moving average + moving Modified utilization moving average + moving

standard deviation < utilization thresholdstandard deviation < utilization threshold Moving average modified to account for fewer drives Moving average modified to account for fewer drives

and extra parity updatesand extra parity updates


19

ImplementationImplementation

Prototyped in Linux 2.6.5Prototyped in Linux 2.6.5 Open source, software RAIDOpen source, software RAID

Implemented block I/O handler, monitor, Implemented block I/O handler, monitor, disk managerdisk manager

Implemented user admin tool to configure Implemented user admin tool to configure devicedevice

Updated Raid Tools to recognize PARAID Updated Raid Tools to recognize PARAID levellevel


20


ChallengesChallenges Prototyping PARAIDPrototyping PARAID Commercial machinesCommercial machines Conceptual barriersConceptual barriers Benchmarks designed to measure peak Benchmarks designed to measure peak

performanceperformance Trace replayTrace replay Time consumingTime consuming


21


multimeter

USB cableclient

server

powersupply

12v & 5vpower lines

powermeasurement

probes

SCSIcable

crossover cable

Xeon 2.8 Ghz, 512 MB RAM36.7 GB 15k RPM SCSI

P4 2.8 Ghz, 1 GB RAM160 GB 7200 RPM SATA

RAID

RAID

RAID

RAID

RAID

BOOT


Measurement frameworkMeasurement framework

22


Three different workloads using two different Three different workloads using two different RAID settingsRAID settings Web trace - RAID level 0 (2-disk gear 1, 5-disk gear 2)Web trace - RAID level 0 (2-disk gear 1, 5-disk gear 2)

Mostly read activityMostly read activity Cello99 - RAID level 5 (3-disk gear 1, 5-disk gear 2)Cello99 - RAID level 5 (3-disk gear 1, 5-disk gear 2)

I/O-intensive workload with writesI/O-intensive workload with writes PostMark - RAID level 5PostMark - RAID level 5

Measure peak performance and gear shifting overheadMeasure peak performance and gear shifting overhead

Speed up trace playbackSpeed up trace playback To match hardwareTo match hardware Explore range of speed up factors and power savingsExplore range of speed up factors and power savings


23

Web TraceWeb Trace

UCLA CS Dept Web Servers (8/11/2006 – 8/14/2006)UCLA CS Dept Web Servers (8/11/2006 – 8/14/2006) File system: ~32 GB (~500k files)File system: ~32 GB (~500k files) Trace replay: ~95k requests with ~4 GB data (~260 MB unique)Trace replay: ~95k requests with ~4 GB data (~260 MB unique)


0

0.1

0.2

0.3

0.4

0.5

0.6

1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96

hours

GB/hour

24

Web Trace Power SavingsWeb Trace Power Savings

0

10

20

30

40

50

60

0 5 10 15 20 25 30

hours

wattsRAID-0

PARAID-0

0

10

20

30

40

50

60

0 5 10 15 20 25 30

hours

wattsRAID-0

PARAID-0


64x – 60 requests/sec

128x – 120 requests/sec 256x – 240 requests/sec

0

10

20

30

40

50

60

0 5 10 15 20 25 30

hours

wattsRAID-0

PARAID-0

64x - 34%

128x - 28%

256x - 10%

Energy Savings

25

Web Trace LatencyWeb Trace Latency

0

0.2

0.4

0.6

0.8

1

1 10 100 1000 10000 100000

msec

RAID-0

PARAID-0

0

0.2

0.4

0.6

0.8

1

1 10 100 1000 10000 100000

msec

RAID-0

PARAID-0


256x

128x 64x

0

0.2

0.4

0.6

0.8

1

1 10 100 1000 10000 100000

msec

RAID-0

PARAID-0256x - within 2.7%

64x - 240% 80ms vs. 33ms

Overhead

26

Web Trace BandwidthWeb Trace Bandwidth

-20

30

80

130

180

0 5 10 15 20 25 30

hours

MB/secRAID-0

PARAID-0

-20

30

80

130

180

0 5 10 15 20 25 30

hours

MB/secRAID-0

PARAID-0


256x

128x 64x

0

20

40

60

80

100

120

140

160

180

0 5 10 15 20 25 30

hours

MB/secRAID-0

PARAID-0

256x - within 1.3% in high gear

Overhead

27

Cello99 TraceCello99 Trace

Cello99 WorkloadCello99 Workload HP Storage Research LabsHP Storage Research Labs 50 hours beginning on 9/12/199950 hours beginning on 9/12/1999 1.5 million requests (12 GB) to 440MB of unique blocks1.5 million requests (12 GB) to 440MB of unique blocks I/O-intensive with 42% writesI/O-intensive with 42% writes


28

Cello99 Power SavingsCello99 Power Savings

0

10

20

30

40

50

0 10 20 30 40 50

hours

wattsRAID-5

PARAID-5

05

101520253035404550

0 10 20 30 40 50

hours

wattsRAID-5

PARAID-5





0

10

20

30

40

50

0 10 20 30 40 50

hours

wattsRAID-5

PARAID-532x - 13%

64x - 8.2%

128x - 3.5%

Energy Savings

29

Cello99 Completion TimeCello99 Completion Time

0.9

0.92

0.94

0.96

0.98

1

1 10 100 1000 10000 100000

msec

RAID-5

PARAID-5

0.9

0.92

0.94

0.96

0.98

1

1 10 100 1000 10000 100000

msec

RAID-5

PARAID-5


128x

64x 32x

0.9

0.92

0.94

0.96

0.98

1

1 10 100 1000 10000 100000

msec

RAID-5

PARAID-5

32x - 1.8ms,26% slowerdue to time spent in lowgear

Overhead

30

Cello99 BandwidthCello99 Bandwidth

1

10

100

1000

0 500000 1000000 1500000

request number

MB/secRAID-5

PARAID-5

1

10

100

1000

0 500000 1000000 1500000

requests

MB/secRAID-5

PARAID-5


1

10

100

1000

0 500000 1000000 1500000

request number

MB/secRAID-5

PARAID

64x 32x

128x

Overhead

< 1% degra-dation duringpeak hours

31

PostMark BenchmarkPostMark Benchmark

Popular synthetic benchmarkPopular synthetic benchmark Generates ISP-style workloadsGenerates ISP-style workloads Stresses peak read/write performance of storage Stresses peak read/write performance of storage

devicedevice


32

Postmark PerformancePostmark Performance


0

50

100

150

200

1K files, 50K trans 20K files, 50K trans 20K files, 100Ktrans

seconds

RAID-5 PARAID-5 high gear PARAID-5 low-gear

33

Postmark Power SavingsPostmark Power Savings


0

10

20

30

40

50

60

70

80

1 11 21 31 41 51 61 71 81 91 10 111 12 13 14 151 16 171

seconds

wat

ts RAID5

PARAID

34

Related WorkRelated Work

PergamumPergamum EERAIDEERAID RIMACRIMAC Hibernator Hibernator MAIDMAID PDCPDC BlueFSBlueFS


35

Lessons LearnedLessons Learned

Third version of design, early design too Third version of design, early design too complicatedcomplicated

Data alignment problemsData alignment problems Difficult to measure system under normal loadDifficult to measure system under normal load Hard to predict workload transformations due to Hard to predict workload transformations due to

complex system optimizationscomplex system optimizations Challenging to match trace environmentsChallenging to match trace environments


36

ConclusionConclusion

PARAID reuses standard RAID-levels without PARAID reuses standard RAID-levels without special hardware while decreasing their energy special hardware while decreasing their energy use by 34%.use by 34%. Optimized version can save even more energyOptimized version can save even more energy

Empirical evaluation importantEmpirical evaluation important


37

QuestionsQuestions

PARAID: A PARAID: A Gear-ShiftingGear-Shifting

Power-Aware RAIDPower-Aware RAID

ContactContact Andy Wang – Andy Wang – [email protected]

PARAID: A Gear-Shifting Power-Aware RAID

Documents

poweraware raidraid12345

power usage

power transitions

highest gear

frequent power switchingparaid

gear utilization threshold

energy savinguse

energy savingadapt