R.A.I.D. Everything you always wanted to know about RAID but didn't know enough to ask Presented to the APCU by John Donahue October 31, 2009.

R.A.I.D.R.A.I.D.

Everything you always wanted Everything you always wanted to know about RAID but didn't to know about RAID but didn't

know enough to askknow enough to ask

Presented to the APCU by John Donahue

October 31, 2009

What we will coverWhat we will cover

Exactly what is RAID?Exactly what is RAID?How does it work?How does it work?When and where would you use it?When and where would you use it?Hardware and Software implementationsHardware and Software implementationsWhat are some best practices for RAIDWhat are some best practices for RAIDMy recommendations for a workstationMy recommendations for a workstationDemo – how to configure a RAID Demo – how to configure a RAID – (if I can access my servers over the internet)(if I can access my servers over the internet)

What exactly is R.A.I.D.What exactly is R.A.I.D.

The most common meaning today is:The most common meaning today is:– RRedundant edundant AArray of rray of IIndependent ndependent DDisksisks– Originally: Originally: RRedundant edundant AArray of rray of IInexpensive nexpensive DDisksisks

The term was coined in 1987 by Dave Patterson, The term was coined in 1987 by Dave Patterson, U of Calif., Berkeley in one of his projects.U of Calif., Berkeley in one of his projects.Original motivation was to replace large and very Original motivation was to replace large and very expensive mainframe DASD drives by an array expensive mainframe DASD drives by an array of cheaper disk drives – like the type often used of cheaper disk drives – like the type often used in PC’s at that timein PC’s at that timeNow it’s basically connecting a collection of hard Now it’s basically connecting a collection of hard drives in an arrangement that enhances drives in an arrangement that enhances reliability, speed, or both.reliability, speed, or both.

A word to the wiseA word to the wise

It may be obvious, but it’s still worth It may be obvious, but it’s still worth mentioning.mentioning.

RAID is RAID is NOTNOT a substitute for “backups” a substitute for “backups”

It can minimize the immediate impact due It can minimize the immediate impact due to disk failures, but it cannot recover data to disk failures, but it cannot recover data due to other failures and disasters.due to other failures and disasters.

Offsite backups are still the best insurance Offsite backups are still the best insurance for data recoveryfor data recovery

Most common flavors of RAIDMost common flavors of RAID

RAID-0 Not truly a RAID (no RAID-0 Not truly a RAID (no RRedundant drive)edundant drive)– Used for simply striping data across multiple disksUsed for simply striping data across multiple disks– Possible performance gain, but Possible performance gain, but nono fault tolerance fault tolerance

RAID-1 Simple mirroring, (2 disks) single fault tolerantRAID-1 Simple mirroring, (2 disks) single fault tolerant– Fastest write performanceFastest write performance– Highest (50%) overhead in storageHighest (50%) overhead in storage

RAID-5 Striping w/parity, (3 or more) single fault tolerantRAID-5 Striping w/parity, (3 or more) single fault tolerant– Slower write performanceSlower write performance– Lowest overhead in storage (33% or less, w/5 disks only 20%)Lowest overhead in storage (33% or less, w/5 disks only 20%)

RAID-6 Striping w/parity, double fault tolerantRAID-6 Striping w/parity, double fault tolerant– More $$$ and used only for exceptionally high reliabilityMore $$$ and used only for exceptionally high reliability

RAID-10 (or 1+0) is mirroring plus stripingRAID-10 (or 1+0) is mirroring plus striping

Less common flavors of RAIDLess common flavors of RAID

JBOD is not RAID, but sometimes mentioned JBOD is not RAID, but sometimes mentioned – JJust a ust a BBunch unch OOf f DDisks strung together to look like a isks strung together to look like a

single volume. a.k.a. Spanned volumesingle volume. a.k.a. Spanned volume– No performance gain or reliability protectionNo performance gain or reliability protection

RAID-2 never caught on (complex, expensive)RAID-2 never caught on (complex, expensive)RAID-3 and 4 Dedicated drive for parity RAID-3 and 4 Dedicated drive for parity (bottleneck on parity drive- rare)(bottleneck on parity drive- rare)RAID-7 not an industry standard- very rareRAID-7 not an industry standard- very rareRAID-01 (or 0+1) and 50, 60. RAID-01 (or 0+1) and 50, 60. – They are similar in concept to RAID-10 but less They are similar in concept to RAID-10 but less

common. All are variations of mirroring and striping.common. All are variations of mirroring and striping.

How It Works, cont.How It Works, cont.

RAID-0 (striping)RAID-0 (striping)

Disk 1 Disk 2 Disk 3 Disk 4

Block a Block b Block c Block d

Block f Block g Block h Block j

Disk 5

Block k Block l Block n Block o

Block e

Block i

Block m


RAID-1 MirroringRAID-1 Mirroring

Disk 1 Disk 2

Block a Block a

Block b Block b

Block c Block c

Block d

Block e

Block d

Block e


RAID 5 Striping with parityRAID 5 Striping with parity


Block a Block b Block c Block d ECC 1

Block e Block f Block g Block hECC 2

Disk 5

Block i Block j ECC 3 Block k Block l

Block m ECC 4 Block n Block pBlock o

ECC 5 Block q Block r Block tBlock s

How It Works, cont.How It Works, cont.RAID-10 or 1+0 (mirroring with striping)RAID-10 or 1+0 (mirroring with striping)



Block e Block f Block g Block h



Block e Block f Block g Block h

mirror

RAID Comparison TableRAID Comparison TableRAIDRAID Min Min

drvsdrvsData Data prot.prot.

Read Read perf.perf.

Write Write perf.perf.

Cap. Cap. Avail.Avail.

Usage commentsUsage comments

00 22 nono highhigh highhigh 100%100% High end workstations, High end workstations, very transitory datavery transitory data

11 22 yesyes highhigh Med.Med. 50%50% Operating systems, Operating systems, transaction databasestransaction databases

55 33 yesyes highhigh lowlow 67%-67%-94%94%

Data warehousing, web Data warehousing, web serving, archivingserving, archiving

66 44 yes 2yes 2 highhigh lowlow 50%-50%-88%88%

high avail. Solutions, high avail. Solutions, servers w/large capcty.servers w/large capcty.

1010 44 yesyes highhigh Med.Med. 50%50% Fast databases, Fast databases, application serversapplication servers

When and Where would you want When and Where would you want to use RAIDto use RAID

In very general terms, let’s consider the impacts In very general terms, let’s consider the impacts of a disk failure.of a disk failure.If a single workstation has a disk failure, you If a single workstation has a disk failure, you probably have one worker unable to perform probably have one worker unable to perform their tasks.their tasks.If a server has a disk failure, you very likely If a server has a disk failure, you very likely could have hundreds or many thousands of could have hundreds or many thousands of workers unable to perform their tasks.workers unable to perform their tasks.Any place where a disk failure would have a very Any place where a disk failure would have a very large impact is a candidate for RAID.large impact is a candidate for RAID.

When and Where would you want When and Where would you want to use RAID cont.to use RAID cont.

Servers (Web, storage, database, email, Servers (Web, storage, database, email, application, etc.)application, etc.)High reliability workstationsHigh reliability workstationsProcess control computersProcess control computersAny place where downtime due to disk Any place where downtime due to disk failures would be unacceptablefailures would be unacceptableWhere higher disk performance provided Where higher disk performance provided by striping is important.by striping is important.Just because you want to play with itJust because you want to play with it

How Is RAID ImplementedHow Is RAID Implemented

Hardware (preferred)Hardware (preferred)– Motherboard or SCSI, SATA, PATA controller Motherboard or SCSI, SATA, PATA controller

card handles all of the details of spanning card handles all of the details of spanning and/or fault toleranceand/or fault tolerance

– This option is generally better, but more $$$This option is generally better, but more $$$

SoftwareSoftware– Operating system handles the details of Operating system handles the details of

spanning and/or fault tolerancespanning and/or fault tolerance– This option should be considered if hardware This option should be considered if hardware

option not available or constrained by $$$option not available or constrained by $$$

Hardware RAIDHardware RAID

Some newer motherboards have RAID supportSome newer motherboards have RAID supportServer controller cards provide RAID ($$$)Server controller cards provide RAID ($$$)– Compaq (SCSI and Fiber Channel)Compaq (SCSI and Fiber Channel)

Hot-pluggableHot-pluggableHot spareHot spare

– Hewlett PackardHewlett Packard– AdaptecAdaptec– LSI MegaRAIDLSI MegaRAID

Workstation SATA controllers (less $$$)Workstation SATA controllers (less $$$)– Options and prices change daily. Check out FRY’s Options and prices change daily. Check out FRY’s

for starters. I have had good results with SIIG and for starters. I have had good results with SIIG and heard good things about Highpoint controllers. heard good things about Highpoint controllers.

Operating Systems That Support a Operating Systems That Support a Software RAIDSoftware RAID

Apple – RAID- 0, 1, 5, 10 Apple – RAID- 0, 1, 5, 10 FreeBSD – RAID-0, 1, 3, FreeBSD – RAID-0, 1, 3, Linux – RAID-0, 1, 4, 5, 6 Linux – RAID-0, 1, 4, 5, 6 Microsoft server- RAID-0, 1, 5 Microsoft server- RAID-0, 1, 5 Windows XP Professional – RAID-0, JBODWindows XP Professional – RAID-0, JBOD– A patch exists to add RAID-1, 5 A patch exists to add RAID-1, 5

Windows 7 Pro – RAID-0, 1 (based on beta)Windows 7 Pro – RAID-0, 1 (based on beta)NetBSD – RAID-0, 1, 4, 5, 10NetBSD – RAID-0, 1, 4, 5, 10– with software RAIDframewith software RAIDframe

OpenBSD - aims to support RAID-0, 4, 5OpenBSD - aims to support RAID-0, 4, 5– with software SoftRAIDwith software SoftRAID

OpenSolaris and Solaris 10 – RAID-0, 1, 5, 6OpenSolaris and Solaris 10 – RAID-0, 1, 5, 6

Best Practices for RAIDBest Practices for RAID

For servers, you For servers, you alwaysalways want RAID and the want RAID and the preferred type of RAID is dependent on the preferred type of RAID is dependent on the server function.server function.Web, and many types of app serversWeb, and many types of app servers– RAID-1 for operating systemRAID-1 for operating system– RAID-5 for everything elseRAID-5 for everything else

Database and write intensive serversDatabase and write intensive servers– RAID-1 for operating systemRAID-1 for operating system– RAID-1 databaseRAID-1 database

Note: RAID 6 or 10 is also a possible option if Note: RAID 6 or 10 is also a possible option if the added reliability is worth the extra $$$.the added reliability is worth the extra $$$.

Best Practices for RAID, cont.Best Practices for RAID, cont.

For workstations For workstations – RAID-1 for operating systemRAID-1 for operating system– RAID-1 for high performance on work disksRAID-1 for high performance on work disks

If multiple disks and performance are more If multiple disks and performance are more important than fault tolerance on work disks, important than fault tolerance on work disks, consider striped RAID-0consider striped RAID-0

– RAID-5 for large local storage (not heavy write RAID-5 for large local storage (not heavy write activity)activity)

My Recommendations for a My Recommendations for a WorkstationWorkstation

If you have RAID option in your motherboard, use it for If you have RAID option in your motherboard, use it for your operating system drivesyour operating system drivesIf you don’t have and don’t want to buy hardware RAID, If you don’t have and don’t want to buy hardware RAID, consider Windows 7 Pro software RAID-1consider Windows 7 Pro software RAID-1Depending on your needs, you might also consider a Depending on your needs, you might also consider a RAID-0 where disk performance is more important than RAID-0 where disk performance is more important than fault tolerance.fault tolerance.Consider RAID-5 for disk storage where you want fault Consider RAID-5 for disk storage where you want fault tolerance but it does not have a high write/change tolerance but it does not have a high write/change activityactivityFor more archival type disk storage, consider a network For more archival type disk storage, consider a network connected disk storage device that uses RAID-5.connected disk storage device that uses RAID-5.

Network Attached RAIDNetwork Attached RAID

Buffalo LinkStation supports RAID 0, 1, 5, 10 with (4) 500GB drives

also gigabit Ethernet connection

This 2 terabyte version sold at Fry’s for $400 in the last August flyer

Demo how to configure RAIDDemo how to configure RAID

Hardware configuration on a Compaq Hardware configuration on a Compaq server using the RAID configuration tool.server using the RAID configuration tool.

Software configuration on a Windows Software configuration on a Windows Server 2003Server 2003

Software configuration on a Windows 7 Software configuration on a Windows 7 Pro (or Ultimate) workstationPro (or Ultimate) workstation

Questions?Questions?

How RAID Parity WorksHow RAID Parity Works

Drive 1 0110 value 6



Drive 4 0101 ECC

In this case, Drive 4 happens to contain the parity for the actual data in the stripe that was written onto Drives 1,2,3.

If any one of the drives 1, 2, 3 fail, the data that was on that drive can be recovered by doing an XOR on the data from the remaining drives.

In this example let’s say drive 2 fails. Then:

0110 From drive 1

1001 From drive 3

0101 From drive 4

-------- XOR

1010 Recovered drive 2 data

The trick is done via “XOR” (Exclusive Or)’s

Additional TidbitsAdditional Tidbits

w/software RAID you may have to use a floppy, w/software RAID you may have to use a floppy, CD, or USB to boot to the second plex of the CD, or USB to boot to the second plex of the system mirror.system mirror.

Software RAID will force a rebuild if proper Software RAID will force a rebuild if proper shutdown was not usedshutdown was not used

Rebuild time can be considerableRebuild time can be considerable– Very rough estimates are 10-15 min. per gigabyteVery rough estimates are 10-15 min. per gigabyte– 1 TB hardware RAID-5 (Buffalo) took several days1 TB hardware RAID-5 (Buffalo) took several days– 30 GB hardware RAID-1 took over 7 hours (software)30 GB hardware RAID-1 took over 7 hours (software)– 120 GB software RAID-1 took over 15 hours (hardwr)120 GB software RAID-1 took over 15 hours (hardwr)

RAID 1+0 vs. 0+1RAID 1+0 vs. 0+1RAID 0+1:RAID 0+1: We stripe together drives 1, 2, 3, 4 and 5 into RAID 0 stripe set We stripe together drives 1, 2, 3, 4 and 5 into RAID 0 stripe set "A", and drives 6, 7, 8, 9 and 10 into RAID 0 stripe set "B". We then mirror A "A", and drives 6, 7, 8, 9 and 10 into RAID 0 stripe set "B". We then mirror A and B using RAID 1. If one drive fails, say drive #2, then the entire stripe set and B using RAID 1. If one drive fails, say drive #2, then the entire stripe set "A" is lost, because RAID 0 has no redundancy; the RAID 0+1 array "A" is lost, because RAID 0 has no redundancy; the RAID 0+1 array continues to chug along because the entire stripe set "B" is still functioning. continues to chug along because the entire stripe set "B" is still functioning. However, at this point you are reduced to running what is in essence a However, at this point you are reduced to running what is in essence a straight RAID 0 array until drive #2 can be fixed. If in the meantime drive #9 straight RAID 0 array until drive #2 can be fixed. If in the meantime drive #9 goes down, you lose the entire array. goes down, you lose the entire array. RAID 1+0:RAID 1+0: We mirror drives 1 and 2 to form RAID 1 mirror set "A"; 3 and 4 We mirror drives 1 and 2 to form RAID 1 mirror set "A"; 3 and 4 become "B"; 5 and 6 become "C"; 7 and 8 become "D"; and 9 and 10 become "B"; 5 and 6 become "C"; 7 and 8 become "D"; and 9 and 10 become "E". We then do a RAID 0 stripe across sets A through E. If drive become "E". We then do a RAID 0 stripe across sets A through E. If drive #2 fails now, only mirror set "A" is affected; it still has drive #1 so it is fine, #2 fails now, only mirror set "A" is affected; it still has drive #1 so it is fine, and the RAID 1+0 array continues functioning. If while drive #2 is being and the RAID 1+0 array continues functioning. If while drive #2 is being replaced drive #9 fails, the array is fine, because drive #9 is in a different replaced drive #9 fails, the array is fine, because drive #9 is in a different mirror pair from #2. Only two failures in the same mirror set will cause the mirror pair from #2. Only two failures in the same mirror set will cause the array to fail, so in theory, five drives can fail--as long as they are all in array to fail, so in theory, five drives can fail--as long as they are all in different sets--and the array would still be fine. different sets--and the array would still be fine.

RAID 1+0 vs. 0+1, cont.RAID 1+0 vs. 0+1, cont.Clearly, RAID 1+0 is more robust than RAID 0+1. Now, if the controller running RAID 0+1 were smart, when drive #2 failed it would continue striping to the other four drives in stripe set "A", and if drive #9 later failed it would "realize" that it could use drive #4 in its stead, since it should have the same data. This functionality would theoretically make RAID 0+1 just as fault-tolerant as RAID 1+0. Unfortunately, most controllers aren't that smart. It pays to ask specific questions about how a multiple RAID array implementation handles multiple drive failures, but in general, a controller won't swap drives between component sub-arrays unless the manufacturer of the controller specifically says it will.

The same impact on fault tolerance applies to rebuilding. Consider again the example above. In RAID 0+1, if drive #2 fails, the data on five hard disks will need to be rebuilt, because the whole stripe set "A" will be wiped out. In RAID 1+0, only drive #2 has to be rebuilt. Again here, the advantage is to RAID 1+0.

Backup Slides if Internet Backup Slides if Internet Connection is Not Available to Connection is Not Available to

My ServersMy Servers

Windows 2003 ServerWindows 2003 Server

Windows 7 betaWindows 7 beta

Compaq Hardware RAID ConfiguratorCompaq Hardware RAID ConfiguratorShowing Physical Disk ViewShowing Physical Disk View

Compaq Hardware RAID ConfiguratorCompaq Hardware RAID ConfiguratorShowing “Logical Volume” viewShowing “Logical Volume” view

Notice that Array A, which is really three physical disks, will appear as a single physical disk to the operating system and will contain the C: drive for the Windows operating system.

The next slide shows this in the Windows Disk Management.

Windows Disk Management View of Compaq RAID arraysWindows Disk Management View of Compaq RAID arrays

Notice “Disk 2” (drive C) appears as a single physical drive to Windows

R.A.I.D. Everything you always wanted to know about RAID but didn't know enough to ask Presented to the APCU by John Donahue October 31, 2009.

Documents