More RAID 5 Advice Needed

[sixdoubleo]

I have 3 Maxtor DiamondMax 9 250GB ATA-133 drives sort of sitting around. Anyway, I was thinking I might buy 2 or 3 more of them, a decent IDE RAID-5 controller and throw the whole deal in my file server so I'd have about 1TB.

I'd like opinions. I'm OK with using Linux..even might consider Linux Software RAID, although I'd prefer hardware RAID.

At some point I would like to build a SATA-based RAID with 300 or 400GB drives when prices come down. Right now I dont want to spend that much. So for all intents and pruposes, this 250GB IDE-based RAID will be temporary...maybe migrate it to my backup server at some point down the road after I build a bigger RAID.

I was looking at the Highpoint Rocket 4xx series.... So what is the basic problem with using IDE-based RAID? As I understand it, there is still an issue with Master/Slave and if a master dies it brings down the channel?

Would I be better of just saying forget it and going SATA? Just that I already have these 3 drives and wanted to take advantage of that by building a semi-cheap RAID.

Ideas?

[dadams]

One possibility is to use Highpoint's RocketRaid 1640. It will use IDE drives if you buy some of the Highpoint connectors that allow you to connect IDE drives to the SATA connections. I haven't used them, but I do use the 1640 with SATA drives and it works pretty well.

Dennis

[sixdoubleo]

Originally Posted by dadams

One possibility is to use Highpoint's RocketRaid 1640. It will use IDE drives if you buy some of the Highpoint connectors that allow you to connect IDE drives to the SATA connections. I haven't used them, but I do use the 1640 with SATA drives and it works pretty well.

Dennis

What advantage would that have over using a RocketRaid 454, which would support IDE drives directly?

[sixdoubleo]

Is there an easy way to identify which cards are true hardware RAID implementations vs. those which are software "assisted". I downloaded the manual for the Highpoint RocketRAID 454 and it uses a Management Service which must run on the host machine. Not sure if I like that.

[mdmint]

Originally Posted by sixdoubleo

I have 3 Maxtor DiamondMax 9 250GB ATA-133 drives sort of sitting around. Anyway, I was thinking I might buy 2 or 3 more of them, a decent IDE RAID-5 controller and throw the whole deal in my file server so I'd have about 1TB.

I'd like opinions. I'm OK with using Linux..even might consider Linux Software RAID, although I'd prefer hardware RAID.

At some point I would like to build a SATA-based RAID with 300 or 400GB drives when prices come down. Right now I dont want to spend that much. So for all intents and pruposes, this 250GB IDE-based RAID will be temporary...maybe migrate it to my backup server at some point down the road after I build a bigger RAID.

I was looking at the Highpoint Rocket 4xx series.... So what is the basic problem with using IDE-based RAID? As I understand it, there is still an issue with Master/Slave and if a master dies it brings down the channel?

Would I be better of just saying forget it and going SATA? Just that I already have these 3 drives and wanted to take advantage of that by building a semi-cheap RAID.

Ideas?

I stay away from those Highpoint controllers for RAID5 for a number of reasons. Primarily never want to use a master/slave configuration in RAID5 unless you like to double your risk of loosing drive access and of total data loss. Since they support it they're cutting corners IMO. In fact Adaptec, Promise & 3ware ATA RAID controllers that support RAID5 don't even allow M/S configurations IRRC. Also far as I can tell the Highpoints don't support OCE.

BTW, nothing at all wrong with using PATA drives for RAID5 arrays, been doing it for years. The challenge is finding a controller with cache, good number of channels and OCE. The 3ware Escalade 7500 series handles most number ATA drives, up to 12 channels/drives, but doesn't support OCE. Promise maxes out at 6 channels and also no OCE. Adaptec has ATA RAID controller supporting OCE but maxes out at 4 channels/HDs. OCE not important if maxing out HDs initial configuration.

Many more good options for OCE and number of HDs with newer SATA RAID5 controllers than older designed ATA RAID5 controllers. That would be the advantage of going with one of them and using PATA to SATA adapters for those HDs your already have. You could even add SATA drives later though if in the same array the overall array performance would be limited by the PATA drives.

[sixdoubleo]

Originally Posted by mdmint

I stay away from those Highpoint controllers for RAID5 for a number of reasons. Primarily never want to use a master/slave configuration in RAID5 unless you like to double your risk of loosing drive access and of total data loss. Since they support it they're cutting corners IMO. In fact Adaptec, Promise & 3ware ATA RAID controllers that support RAID5 don't even allow M/S configurations IRRC. Also far as I can tell the Highpoints don't support OCE.

BTW, nothing at all wrong with using PATA drives for RAID5 arrays, been doing it for years. The challenge is finding a controller with cache, good number of channels and OCE. The 3ware Escalade 7500 series handles most number ATA drives, up to 12 channels/drives, but doesn't support OCE. Promise maxes out at 6 channels and also no OCE. Adaptec has ATA RAID controller supporting OCE but maxes out at 4 channels/HDs. OCE not important if maxing out HDs initial configuration.

Many more good options for OCE and number of HDs with newer SATA RAID5 controllers than older designed ATA RAID5 controllers. That would be the advantage of going with one of them and using PATA to SATA adapters for those HDs your already have. You could even add SATA drives later though if in the same array the overall array performance would be limited by the PATA drives.

Hmmm, the thing is I was trying to get away somewhat cheap with this arrray...since in about 12-18 months it'll probably be in my backup server...controller card and drives as a unit. Then I'll build a new array from 300 or 400GB drives (with a SATA-based controller).

The 7506-4LP would be about in my price range, but with only 4 drives, I'd have only 750GB. That aint much.... One question: Does the 7506-4LP allow for a 5th drive as a spare? Or if I use a spare, would I eat up one of my 4 drives?

I totally hear what you're saying though about the master/slave setup. If the channel itself dies, you just lost 2 drives. However, it's not usually channels that die, it's drives that die. I also have a problem with these software assisted controllers. Might mean I can't drop it in my Fedora Core box later, if there is no driver....

[mdmint]

Originally Posted by sixdoubleo

Hmmm, the thing is I was trying to get away somewhat cheap with this arrray...since in about 12-18 months it'll probably be in my backup server...controller card and drives as a unit. Then I'll build a new array from 300 or 400GB drives (with a SATA-based controller).

The 7506-4LP would be about in my price range, but with only 4 drives, I'd have only 750GB. That aint much.... One question: Does the 7506-4LP allow for a 5th drive as a spare? Or if I use a spare, would I eat up one of my 4 drives?

Unfortunately good expandable RAID5 storage and cheap don't necessarily mix. 4 channels equals 4 drives, no 5th for online spare. So it'd be 3 drive RAID5 array with online spare or 4 drive array no online spare. I know what you mean 750GB ain't much! I originally designed my SageServer with 4port controller 750GB 4 drive RAID5, month later upgraded whole subsystem to 1.495TB 7 drive RAID5 w/online spare for main Video storage AND 500GB of original 4 drive 750GB RAID5 allocated for Library.

I totally hear what you're saying though about the master/slave setup. If the channel itself dies, you just lost 2 drives. However, it's not usually channels that die, it's drives that die.

True, the channel on the controller isn't usually the issue. But remember in a master/slave configuration if the master HD dies so does the slave, so both gone from the array. I don't recall what happens to the master if the slave dies. And in RAID5 loosing two drives at the same time in any manner means ALL data is history.

I also have a problem with these software assisted controllers. Might mean I can't drop it in my Fedora Core box later, if there is no driver....

[sixdoubleo]

Originally Posted by mdmint

But remember in a master/slave configuration if the master HD dies so does the slave, so both gone from the array. I don't recall what happens to the master if the slave dies. And in RAID5 loosing two drives at the same time in any manner means ALL data is history.

You sure about that? That's news to me. I think I've had a "Master" CD-ROM die and the secondary device still worked. Hmm....

[mdmint]

Originally Posted by sixdoubleo

You sure about that? That's news to me. I think I've had a "Master" CD-ROM die and the secondary device still worked. Hmm....

You may be right and it's a "depends" on type of failure. If it's the PCB on the master HD that fails then since it's the actual direct HD controller both M&S won't work IRRC. (And PCB quite often the cause of a HD failure. Often you can pull a good PCB and put it on the failed drive and it's good to go, assuming identical HD model and revision.)

[sixdoubleo]

UPDATE: Highpoint 454 Mini-Review

So I started building my new file server (2003 Server, Active Directory, and data storage server) and decided to try one of these Highpoint RocketRAID 454 controllers. Here's a brief summary of my experience with it.

Hardware:
ECS K7S5A Motherboard (SiS 735 Chipset)
AMD Duron 1300
256MB PC-2700 RAM
1 80GB Seagate ATA-100 (for system drive)
6 250GB Maxtor DiamondMax ATA-133 Drives (for RAID 5)
Old ATI GII-460 w/8MB RAM)
Netgear GA311 Gigabit Adapter
Cooler Master CM Staker Case
Thermaltake 420W Power Supply


Despite what others have warned against (stanger, mdmint, et al.) I just had to see for myself just how BAD the Highpoint RocketRAID 454 was. I figured for $89 it was worth a try and my curisosity was getting the better of me. If I didn't like it, I'd just order a good controller.

I went with 6 250GB Maxtor ATA-133 drives. 3 were the DiamondMax 9 series drives that I already owned and the then I ordered three new ones (DiamondMax 10's) to get a total of six.

So 6 250GB drives in a RAID 5 array comes to roughly 1.1-1.25TB (depending on how you calculate it).

First off, I mounted the drives in a Cooler Master CM Stacker case which has its own 4-in-3 cage (very nice one as a matter of fact). The remaining drives just went into a cheapo I-star 3-in-2 mount. Once I determine that this thing is good to go for "production", I'll order another CoolerMaster 4-in-3 mount.

As has been stated, the HP 454 uses 4 IDE channels with 2 drives per channel, netting a total of 8 drives possible. So I had 4 drives on 2 channels, and then 2 drives on their own channel (as recommended by Highpoint's drive layout chart).

The BIOS setup utility on the Highpoint is actually very intuitive and simple. I selected all 6 drives, selected RAID 5 and then told it to do a "Zero Build" which is where it writes 0's to the drives and establishes parity. Well, this process took 5 hours. I thought that was a little long.

Once that was done, I booted into Windows 2003 Server (I already had it installed on an 80GB system drive) and installed the drivers for the RAID controller. This went well too. I downloaded the latest 2k3 driver from Highpoint's website.

After goofing around with how to format the array (physial partition, dynamic disk, simple volume, etc, etc) I found that the only way to get this thing to format is to convert it to a dynamic disk and create a simple volume. If you treat it like a normal hard drive and create a Primary partition, it wont format. It'd get to the end of the format and Windows will simply say "The format did not complete". Anyway, as I said, you have to convert it to a dynamic disk and create a simple volume. This was not documented at all in the Highpoint manual. I felt they should have given a recommended formatting procedure at least.

Anyway, since I didn't want to wait for the whole 1TB to format I just did a quick 8GB volume so I could test the performance of this thing.

Let me just say that the performance of this thing iks AWFUL. Any type of access to the RAID array sends the CPU to 100%....pegs it. The system becomes very choppy and you can't use the mouse or anything else.

Copy data to this thing was SLOOOOW. Just a simple comparison... Copying a 1GB file over the network to the C: Drive (the 80GB Seagate) took 48 seconds. The same copy over the network (gigabit) to the D: drive (the RAID 5 array) took 2:45 (2 minutes, 45 seconds).

On top of the slow performance, it was very unstable. Several times I rebooted and it would come back and tell me that the RAID was inconsisted and that I should rebuild it. Incidentally, going in to rebuilt it, it would estimate the time at 50 hours!!!! (yes, fifty hours, not five hours).

So anyway, if I were to read something like this my immediate thought would be "bah...this sixdoubleo guy doesn't know what he's doing." I wont comment on that except to say that I have built literally hundreds of systems, work with SCSI RAID daily, and this was a very clean and stable system. I went to great lengths to pre-install a very clean 2003 server and get it very stable and "snappy" before introducing the RAID controller into the mix. I ghosted the system prior to putting the RAID controller in just so I'd have a stable base to refer back to.

Nonetheless, the Highpoint is now boxed back up and ready to ship back. In its place, I have ordered a 3Ware 7506-8. When I get that controller in my hands I'll post my experience with setting it up and what not.

So yea, Highpoint RAID controller = bad news.... Its low performance alone is enough to negate any cost savings.

[mdmint]

[mikesm]

Sorry to hear your performance sucks. Windows software RAID isn't known for it's speed. You should go to a raidcore hardware controller if you have to use windows, or Linux software RAID would do much much better, but don't go there if you haven't used linux before.

I have seen reports of folks using the Asus A8N Nforce4 MB with 8 SATA ports (4 on the high speed Nforce4 and 4 on a 66 Mhz PCI connection) with outstanding speed under Linux software raid5. But this is a hefty CPU and MB... No I/O bottlenecks though...

Thanks,
Mike

[sixdoubleo]

Originally Posted by mdmint

Go ahead...laugh it up. I was expecting that.

But hey...I'm sure you know how it is. You never just TRUST what anybody says. Gotta see for yourself.

Nonetheless, the Escalade will be here tomorrow afternoon and I can't wait. Itching to get my file server back up and running.

So just out of curisosity, what can I expect as far as build time for the array? 6 250GB drives....

[mdmint]

Originally Posted by sixdoubleo

Go ahead...laugh it up. I was expecting that.

But hey...I'm sure you know how it is. You never just TRUST what anybody says. Gotta see for yourself.

Nonetheless, the Escalade will be here tomorrow afternoon and I can't wait. Itching to get my file server back up and running.

So just out of curisosity, what can I expect as far as build time for the array? 6 250GB drives....

Dang, I honestly don't remember how long it took to setup my 7x250GB RAID5 SATA +online spare Escalade array inc. fdisk and format 5GB OS partition (wish I'd made it 10GB, had to get creative to have enough space to go from WinXP sp1 to sp2). Fdisk is stupid and will first scan the entire logical drive, then when you say no to using it all for Primary scan the entire logical drive again before letting you tell it how much to use. Always thought that was stupid of MS. 1.495TB partition I setup using WinXP disk mgr, 64kb clusters, quick format. That part took no time at all.

If you're using separate OS drives then setting up your array and NT formatting a breeze.

(Typed this during final reboot of my Server after pulling one of my PVR250 and replacing with PVR500MCE for 5 tuners in Server plus HTPC with networked PVR250 for digital cable. Not sure what I'm going to do with the PVR250 I just pulled...ok, I don't type that fast or my Server boot that slow, finish booting a bit ago... )

[Blue]

Originally Posted by mikesm

You should go to a raidcore hardware controller if you have to use windows

I could be wrong, but I'm pretty sure the RAIDCore (aka Broadcom) SATA controllers are also software based.

[sixdoubleo]

Originally Posted by mdmint

Dang, I honestly don't remember how long it took to setup my 7x250GB RAID5 SATA +online spare Escalade array inc. fdisk and format 5GB OS partition (wish I'd made it 10GB, had to get creative to have enough space to go from WinXP sp1 to sp2). Fdisk is stupid and will first scan the entire logical drive, then when you say no to using it all for Primary scan the entire logical drive again before letting you tell it how much to use. Always thought that was stupid of MS. 1.495TB partition I setup using WinXP disk mgr, 64kb clusters, quick format. That part took no time at all.

If you're using separate OS drives then setting up your array and NT formatting a breeze.

(Typed this during final reboot of my Server after pulling one of my PVR250 and replacing with PVR500MCE for 5 tuners in Server plus HTPC with networked PVR250 for digital cable. Not sure what I'm going to do with the PVR250 I just pulled...ok, I don't type that fast or my Server boot that slow, finish booting a bit ago... )

I always use a separate drive just for the OS. In the event of a RAID or controller problem, I would still be able to boot my system. Which is why I'm surprised that you configured your system partition on your RAID away. Why don't you use a dedicated drive for the system and use the RAID for data only? You definitely don't seem to have a shortage of cash.....go get a system drive! (Or beter yet, mirror two drives for system)




PS - If you can't figure out what to do with the extra PVR-250, I could use a 3rd.

[sixdoubleo]

Originally Posted by mikesm

Sorry to hear your performance sucks. Windows software RAID isn't known for it's speed. You should go to a raidcore hardware controller if you have to use windows, or Linux software RAID would do much much better, but don't go there if you haven't used linux before.

I have seen reports of folks using the Asus A8N Nforce4 MB with 8 SATA ports (4 on the high speed Nforce4 and 4 on a 66 Mhz PCI connection) with outstanding speed under Linux software raid5. But this is a hefty CPU and MB... No I/O bottlenecks though...

Thanks,
Mike

I am no stranger to Linux. I currently run a Fedora Core 3 server as my backup server. Everything on the network backs up to it. I'm somewhat curious about how well Linux S/W RAID works and I may play with it a bit just in a test environment.....but I don't think I'm currently ready to trust my live production data to it yet.

As I said, I went ahead and ordered an Escalade card so I should be in good shape there.

[mdmint]

Originally Posted by sixdoubleo

I always use a separate drive just for the OS. In the event of a RAID or controller problem, I would still be able to boot my system. Which is why I'm surprised that you configured your system partition on your RAID away. Why don't you use a dedicated drive for the system and use the RAID for data only? You definitely don't seem to have a shortage of cash.....go get a system drive! (Or beter yet, mirror two drives for system)




PS - If you can't figure out what to do with the extra PVR-250, I could use a 3rd.

My original Sage Server design was for a 4x250GB drive RAID5 for video storage and 2 drive mirror for OS. Within a month I realized the 750MB RAID5 wasn't going to cut it and replaced with 8x250HD with online spare array. The Codegen Mini-Server Tower case I'd chosen was at this point maxed out even using 4in3 cages. Not really taking a performance hit having OS on same array with Sage. I do have a standalone PATA HD in the 3&1/2" FDD slot which I use to GHOST backup the OS partition and is the spare OS boot drive if needed. Prefer to use it as backup OS drive since I have it power down in power management while RAID drives can't.

BTW, yeah at the time when designing and building my Sage system cash wasn't really an issue. We were DINKS with both having long established JOBs. Since then after almost 14yrs with HP my position "was affected by out-sourcing" as they put it so cash flow a bit tighter at the moment...

I'll likely be offering that extra PVR250 here (Frey Community Forum) before eBay.

[BobPhoenix]

Originally Posted by sixdoubleo

Copy data to this thing was SLOOOOW. Just a simple comparison... Copying a 1GB file over the network to the C: Drive (the 80GB Seagate) took 48 seconds. The same copy over the network (gigabit) to the D: drive (the RAID 5 array) took 2:45 (2 minutes, 45 seconds).

On top of the slow performance, it was very unstable. Several times I rebooted and it would come back and tell me that the RAID was inconsisted and that I should rebuild it. Incidentally, going in to rebuilt it, it would estimate the time at 50 hours!!!!

I have both a 454 and a 464. The 464 is supposed to be a hardware Raid5 solution the 454 is software - saw review of 1820A (suppose to have the same hardware chip) that says it is software so not sure now - tech support said 464 was hardware so maybe review of 1820A was wrong. I don't use the 454 for anything but Raid0 or 1 but I have the 464 as my recording controller. It works fairly well with Raid5 no noticable increase in CPU when recording 4 simultaneous streams. So you would think I would be happy with it wouldn't you! Found the same problem you did with rebuilding the array took 17 hours for a 600 GB (4 200GB drives no spare) array. Also worried about one array taking out the other (I have two arrays set up one on the masters and the other on the slaves) so am going to go with an Escalade 7506-8 or 12 when I get the money. Just crossing my fingers now that it will last until I get the money.

BobP.

[sixdoubleo]

Originally Posted by BobPhoenix

I have both a 454 and a 464. The 464 is supposed to be a hardware Raid5 solution the 454 is software - saw review of 1820A (suppose to have the same hardware chip) that says it is software so not sure now - tech support said 464 was hardware so maybe review of 1820A was wrong. I don't use the 454 for anything but Raid0 or 1 but I have the 464 as my recording controller. It works fairly well with Raid5 no noticable increase in CPU when recording 4 simultaneous streams. So you would think I would be happy with it wouldn't you! Found the same problem you did with rebuilding the array took 17 hours for a 600 GB (4 200GB drives no spare) array. Also worried about one array taking out the other (I have two arrays set up one on the masters and the other on the slaves) so am going to go with an Escalade 7506-8 or 12 when I get the money. Just crossing my fingers now that it will last until I get the money.

BobP.

Yeah, that seems to be the consensus (that the 454 is fine for RAID 0 or 1, but NOT RAID 5). As for the 464, the only difference I can see is that they advertise an "Enhanced XOR formula". Beyond that, I don't know that it is truly hardware or not.

After using the Escalade, I'm not sure I'd bother with the Highpoints. It certainly is in a different class.