Looking for raid system suggestions

[lovingHDTV]

I'm seriously thinking of creating a large Raid5 disk array to store all my music, video, and family pictures. I've an old 400Mhz PII system laying around and was wondering if I could resurect it into a Linux running disk array.

I know that Linux now supports Raid arrays, but don't know if the cpu could handle the compute load for a Raid5 software solution. Anyone know anything different?

So I'm thinking a raid card would be best. Suggestions for a card that is supported via Linux? Hopefully not a $400+ card. I plan on using IDE drives, 1 drive per IDE channel type thing.

I know a couple of you out there have a similar setup. Please help me plan mine.

thanks

[stanger89]

If you're stuck on PATA:
http://www.newegg.com/Product/Produc...82E16816116020

Although since SATA drives aren't really any more expensive, I'd recommend going that route. Also note that that card doesn't support OCE.

[mdmint]

Originally Posted by stanger89

If you're stuck on PATA:
http://www.newegg.com/Product/Produc...82E16816116020

Although since SATA drives aren't really any more expensive, I'd recommend going that route. Also note that that card doesn't support OCE.

Well he did say hopefully not a $400 card and at Newegg with shipping the Escalade 7506-8 only comes to $399.99!

Good question whether planning going PATA or SATA. Cost isn't much different anymore.

Having owned & used both 3Ware & Promise RAID5 solutions I'd say the 3Ware a better product than the Promise. (and still use a Promise FastTrak S150 SX4 in my main workstation) Sure there are cheaper controllers than 3Ware, and you get what you pay for. I've been hammering my Escalade 9500S-8 for 'bout a year and half, no problems.

[lovingHDTV]

I found this article that had an interesting approach.
http://www.finnie.org/terabyte

He used the HighPoint RocketRAID 454 card to build a RAID 50 server. The card supports both master and slave drives on the ide channle. He then built 1 RAID5 array with all primary drives, then 1 RAID5 array on all slave drives, then ran RAID0 on the two arrays. This gives protection on loosing an entire channel.

Unfortionately, I really like the automatic array expansion capability and I've not found any software solutions that can do this. Has anyone else?

I'm not stuck on PATA it is just that I already have 2 of these drives.

thanks,

[salsbst]

Raid 50 for music, video, and family pictures? Tremendous waste. Better to invest the savings in a true backup plan and stick to RAID 5.

Running any RAID array off of slave drives is probably also a bad idea.

[stanger89]

Originally Posted by lovingHDTV

I found this article that had an interesting approach.
http://www.finnie.org/terabyte

He used the HighPoint RocketRAID 454 card to build a RAID 50 server. The card supports both master and slave drives on the ide channle. He then built 1 RAID5 array with all primary drives, then 1 RAID5 array on all slave drives, then ran RAID0 on the two arrays. This gives protection on loosing an entire channel.

I wouldn't touch a Highpoint card with a 10-foot pole, well I would if I was strapping a stick of C4 to it And running drives master and slave scares me quite a bit.

FWIW, my take is if you can't afford (or are unwilling to do it right, ie good Hardware RAID card) then a big array is not for you. They aren't cheap, and you really don't want to skimp on something like that.

Unfortionately, I really like the automatic array expansion capability and I've not found any software solutions that can do this. Has anyone else?

Linux, can, but the OCE is "beta" I believe.

I'm not stuck on PATA it is just that I already have 2 of these drives.

What I did when building my array is investigate about every possible combination, different sized drives, different sized cards. I would break them all down into cost/GB, including the card. Because in the end there 3 things to consider, reliability, and after that cost/GB and total size (with the last two in sort of a give an take).

For me, since I had a great deal on half my 250GB PATA drives, I went that route. You may find that since you have those 2 drives it's a significant savings to build from them, or you may find that it's only marginally more expensive to build from scratch. Remember you could always use those 2 drives as non fault tolerant storage in addition to your array. I've got my Sage recordings on a lone 200GB drive, since nothing stays on there too long.

[sleonard]

How do you backup TB's or even many 100's of GB? I haven't seen any tape systems that can back up that much data except for very expensive enterprise solutions. Backing up to DVD would take forever and you would have to sit there and feed a lot of discs. What else is there? Even Blu-Ray (when it comes out) would require a lot of discs.
I'll have 1.3 TB by next week and 2.3 TB by the end of the year.

[mdmint]

Originally Posted by sleonard

How do you backup TB's or even many 100's of GB? I haven't seen any tape systems that can back up that much data except for very expensive enterprise solutions. Backing up to DVD would take forever and you would have to sit there and feed a lot of discs. What else is there? Even Blu-Ray (when it comes out) would require a lot of discs.
I'll have 1.3 TB by next week and 2.3 TB by the end of the year.

Truth be told I don't backup my 1.5TB RAID5 (w/online spare) main Sage storage & 0.5TB RAID5 Library storage partition on my workstation. Yeah, it's a gamble but as others say it's only recorded or ripped DVD stuff anyway. It would be a major bummer to loose multiple shows entire season records and the Wif would kill me but so far so good! Atleast I did build array with online spare for immediate automatic rebuild in case of HD failure. (We record the entire season of many shows then watch 'em later)

Cost effective wise about the cheapest way to backup a large home array is to another large array.

[stanger89]

Originally Posted by mdmint

FWIW I was impressed by the very decent quality avi 1/10 the size of my Std DVD Q records BTW.)

A good chunk of that (maybe 1/2 or more) is the difference between offline encoding and realtime, hardware constrained encoding. If you've seen digital cable you'll know what I mean. Digital cable (SD) runs between 1-3Mbps, compared to the 6-8Mbps required by the Connexant -416

[lovingHDTV]

I am in the middle of creating my spreadsheet of solutions, trying to figure out if/what I go with. It is nice to have a lot of feedback on others experience.

My backup plan is to have a spare drive in the system, then when one drive dies, go buy another and swap it in. Even if 1 drive a year fails, I'm only out $100ish for back ups. Is this seriously flawed thinking?

[Kanati]

If you have a raid 50 system you don't need backups... well... I suppose if the house burnt down you would. Basically you could lose ummmmmm 10 or 15 drives (can't remember) and still be fine with raid 50. But as stated... Super overkill on the homefront. Makes the geek in me drool at the thought though.

Backing up something like 1.5 Tb is a pain in the butt, but you would definitely be looking at a streaming tape setup, or another hard drive array mirroring it on another machine over a gigabit network connection. Personally I just back up all my shows to dvd when I finish watching them. So I spend a lot of time cutting commercials and burning dvds.

[stanger89]

Originally Posted by Kanati

If you have a raid 50 system you don't need backups... well... I suppose if the house burnt down you would. Basically you could lose ummmmmm 10 or 15 drives (can't remember) and still be fine with raid 50. But as stated... Super overkill on the homefront. Makes the geek in me drool at the thought though.

No offense, but that is totally wrong, two well-placed failures would kill the whole array. I think you're thinking of RAID 5+1, but even with that 4 well placed failures would kill it.

[Kanati]

Actually I was thinking of an array system I helped put together once that contained roughly 50 drives... It was an array of arrays. The fault tolerance on it was such that we could lose two or three of the 5 drive arrays simultaneously and we wouldn't lose data... I had thought that it was raid 50. Upon searching, I see that raid 50 can be done with as little as 5 drives... So I have no idea what the monster we had at work was called.

[mdmint]

Originally Posted by Kanati

Actually I was thinking of an array system I helped put together once that contained roughly 50 drives... It was an array of arrays. The fault tolerance on it was such that we could lose two or three of the 5 drive arrays simultaneously and we wouldn't lose data... I had thought that it was raid 50. Upon searching, I see that raid 50 can be done with as little as 5 drives... So I have no idea what the monster we had at work was called.

Actually minimum of 6 drives for Raid50. Raid0 array comprised of min' two Raid5 arrays and each Raid5 array min' 3 drives. Raid50 Array Capacity: (Size of Smallest Drive) * (Number of Drives In Each RAID 5 Set - 1) * (Number of RAID 5 Sets). As Stanger indicated redundancy limited to 1 drive failure per Raid5 array without loosing data. Purpose of Raid50 isn't so much increased redundancy but increased speed, especially writes, via Raid0 of Raid5s versus straight Raid5. For current Sage purposes even large number of disk Raid5 arrays sufficient speed for many simultaneous tuners recording and concurrent playback streams.

[teknubic]

Somewhere along the line you're gonna run out of power and bays. This is what happened to me when I was looking to take my 1TB raw storage to 1TB RAID. I couldn't physically fit any more in the chasis so I started looking into external solutions. Firewire Depot has multi-drive enclosures on the cheap. The I/O should be plenty for the application but I can't vouch for the noise or heat on these.
HTH

[RandyJohnson]

I just put together a 4TB Raid 5 system (3.5TB usable) for less than $4,000.

8 Hitachi 500GB SATA II drives were used with the new Tekram SATA II Raid controller.

newegg had the best price on these drives.

If you have the option SATA II is the only way to go given the test results seen to date.

[mdmint]

Originally Posted by RandyJohnson

I just put together a 4TB Raid 5 system (3.5TB usable) for less than $4,000.

8 Hitachi 500GB SATA II drives were used with the new Tekram SATA II Raid controller.

newegg had the best price on these drives.

If you have the option SATA II is the only way to go given the test results seen to date.

Sweet The ARC-1120 8 port (or 1220 depending on bus) or did you go higher model and allow for future expansion? I mean, only 3.5TB! Personally with that controller I would have gone for higher double drive failure redundancy Raid6 and measly 3TB. As it is I'm stuck with Raid5 and online spare, no Raid6 with the Escalade 9500S-8 boo-hoo

[lovingHDTV]

mdmint,
with the 9500s-8, does the OCE support different drive sizes? I don't even know if this is possible, but if I start the array with 3 250g drives, then add 3 300g drives will it use all the space on the 300g drives or just the first 250g?

Anyone know?

[stanger89]

Just the first 250, there's no way around it, you're limited by the size of the smallest drive in any RAID array.

[lovingHDTV]

OK, that is what I thought, but was not sure.

I wonder what would happen if you slowly removed the small drives. Would it finally be able to use the who 300G after all the 250G drives were gone? I doubt it, that would be nice.