Let's beat this RAID thing to death

[Spectrum]

I'm fast running out of space now that the fall season is upon us and I have already pulled the trigger on 4 2TB drives to move some of the archivable type stuff off the HTPC and free up space for recordings. My original plan was to put them in my primary desktop using FlexRAID and just share them out but that got busted the more I read into it. I don't like the fact that I'd need have separate volumes and manually run the re-synchs etc.

I started looking into just adding a RAID card that supports RAID 5 + spindown and so far the only ones I've found are the Arecas and the Adaptec 5000's.

Anyone got opinions on those or know of any other hardware RAID controllers that support spindown?

[Fuzzy]

For starters, you don't really 'NEED' hardware RAID for archived media. The biggest advantage of hardware raid over software is the speed difference. If you are genuinly using this for archived media, then you really don't NEED the extra speed (especially if it ends up costing you a LOT more). Windows software RAID5 (Dynamic Disks) is actually pretty good... but unfortunately, they've removed if from windows 7 (only availabe in server 2008 now). It was possible in the past to trick (hack) software RAID5 support into XP by modifying a couple system files, but that's no longer an option in vista/7. A shame, too, since it was a nice, controller agnostic RAID system.

[davidk21770]

I had an Areca raid for a while - an ARC-1220.

1. It benched really fast (I had 5 drives in it), but had problems with interleaved operation so that simultaneous recording and playback caused playback to hesitate significantly every 10 seconds or so. I tried lots of different configurations and they all had the same problem.

2. It only lasted a couple of years before it became flaky and finally died completely. (It never lost data, but the http interface started hanging).

3. If you do go raid, I recommend raid 5. Not all drives will raid. I had problems with a drive now and then and would have lost the whole thing if not for the fact that I could remove the recalcitrant drive, put it back in and rebuild the raid (found that 2 drives were going and finally replaced 'em -- this was not the halting problem, had that problem no matter what drives I used).

Edit: I finally removed the raid and went to several 1+TB drives and haven't had a problem since.

[paulbeers]

In my house, the loss of recordings is not important. Because of that, I don't RAID my recording drives. I just have them all as separate drives. Now my DVD/Blu-Ray drives I do. I use RAID 5 + hot spare (my raid controllers, Dell Perc 5i's do not do RAID 6 otherwise I would have gone that route).

These cards (both the Perc 5i or Perc 6i) are fairly cheap used. They run anywhere from $60 to $120 on ebay and various vendors online. However, these do not allow drive spin down. I think the power usage numbers when idle tend to get a bit over stated. I use green drives in my RAIDs (with no issues as others have reported) and they only use 3 watts of power while idle. That means that I can have my entire Perc 5i card running and not doing anything and still only use 24-25watts of power and never have to wait for the drives to spin up.

I can't really recommend software raid setups. They tend to be motherboard specific, so if your motherboard dies it can be difficult to retain the data. In my case if one of my Perc 5i's dies, I just drop in my spare, import the raid and its ready to rock and roll....'

Just my .02 worth.

[Fuzzy]

Originally Posted by paulbeers

I can't really recommend software raid setups. They tend to be motherboard specific, so if your motherboard dies it can be difficult to retain the data. In my case if one of my Perc 5i's dies, I just drop in my spare, import the raid and its ready to rock and roll....'

Just my .02 worth.

That's why I recommended the Windows Dynamic Disks as a software RAID. It is completely controller/motherboard agnostic. You can even move the set to a completely different system, and as long as that system supports the same RAID level, you can bring it in and activate it. (Well, I'm not sure on Dynamic Disk BACKWARDS compatability.. so a RAID5 from Server 2008 might not be able to be moved to a hacked winxp system). I used to run a mixed up (2 IDE, 2 SATA) 4x400GB RAID5 Dynamic Disc on a hacked XP install for my recordings/imports/everything. Moved it between 3 different motherboards over that time, and it never had a single problem with the transition. If I could do this with win7, I'd still be using Dynamic Disks today (albeit with larger discs).

[paulbeers]

Originally Posted by Fuzzy

That's why I recommended the Windows Dynamic Disks as a software RAID. It is completely controller/motherboard agnostic. You can even move the set to a completely different system, and as long as that system supports the same RAID level, you can bring it in and activate it. (Well, I'm not sure on Dynamic Disk BACKWARDS compatability.. so a RAID5 from Server 2008 might not be able to be moved to a hacked winxp system). I used to run a mixed up (2 IDE, 2 SATA) 4x400GB RAID5 Dynamic Disc on a hacked XP install for my recordings/imports/everything. Moved it between 3 different motherboards over that time, and it never had a single problem with the transition. If I could do this with win7, I'd still be using Dynamic Disks today (albeit with larger discs).

Yeah, I can't really speak to Windows Dynamic Disks as a software RAID and the portability. I've never tried it. Based on what you are stating, it also sounds like it is OS limiting. Not a huge deal, but worth noting. I certainly can not recommend software RAID built into a motherboard. That's just asking for trouble.

[Fuzzy]

Originally Posted by paulbeers

Yeah, I can't really speak to Windows Dynamic Disks as a software RAID and the portability. I've never tried it. Based on what you are stating, it also sounds like it is OS limiting. Not a huge deal, but worth noting. I certainly can not recommend software RAID built into a motherboard. That's just asking for trouble.

For me, OS limiting isn't NEAR as restrictive as hardware limiting.. My OS doesn't really 'fail' like a motherboard or drive controller can. The arrays are forwards compatible, so you can create an array in 2003, and use in 2008 (and most likely any future server OS's. I just wish they were enabled in the desktop (at least ultimate and enterprise) versions. The only thing preventing it is the licensing - the components are there (which is how the hacks were done with XP.. you altered the appropriate .dll files to look for winxp instead of win2k3 to enable RAID5 support).

[Bizarroterl]

RAID5 can be a good solution as long as you keep some caveats in mind.

A high percentage of drive failures occur when drives refuse to spin up. If you're running a RAID5 array constantly the older it gets the greater the chance of this happening - and happening with more than one drive. Controllers that spin down drives help reduce the chances of this type of failure hitting twice at the same time. Powering the array down occasionally does the same thing.

If you run a many drive array a failure replacement can result in literally days of rebuild time - just when the array is the most vulnerable and the drives are being worked hardest. Is the controller/software robust enough to handle a power failure during the rebuild?

If the controller fails in a hardware based array one of two things will happen. It will eat the array or it won't. If it doesn't can you access the array with another controller? Maybe, but keep in mind there are no standards on how arrays are defined so an identical controller may be needed.

In enterprise (business) IT these aren't as much as an issue since the data is actively backed up. In a home media setting backups are typically the original media, so a failed array means a bunch of time re-ripping the originals. Unless you don't have the originals, IE your recordings of the entire Lost program.

[Spectrum]

Fuzzy:
I thought about software RAID but poo-poo'd it on general principle I knew XP did stripes and mirrors but didn't know it could do RAID 5. The box in question is a Win 7 machine so the RAID 5 issue is a moot point.

Paul:
I considered the perc cards but they don't do spindown which is a dealbreaker for me. It's more of a noise issue than a power one, my bedroom/office environment is where the box has to live and I have to live with it. I know the drives won't as loud as the old 10k cheetahs; I have a box of those (a whole 90gigs!!) and they whiiiiiiine when idle and chunk when they access. I had an array of 4 or 5 in an old Sun box that would heat the room up about 5 degrees

What I may do is just set them up as dynamic disks and create a spanned volume. Assuming Windows will let them sleep that may be the best solution and the price is right.

Thanks for the info

[mikejaner]

Originally Posted by Spectrum

Fuzzy:
I thought about software RAID but poo-poo'd it on general principle I knew XP did stripes and mirrors but didn't know it could do RAID 5. The box in question is a Win 7 machine so the RAID 5 issue is a moot point.

Paul:
I considered the perc cards but they don't do spindown which is a dealbreaker for me. It's more of a noise issue than a power one, my bedroom/office environment is where the box has to live and I have to live with it. I know the drives won't as loud as the old 10k cheetahs; I have a box of those (a whole 90gigs!!) and they whiiiiiiine when idle and chunk when they access. I had an array of 4 or 5 in an old Sun box that would heat the room up about 5 degrees

What I may do is just set them up as dynamic disks and create a spanned volume. Assuming Windows will let them sleep that may be the best solution and the price is right.

Thanks for the info

XP can't do RAID 5. What he is talking about hacking in the Server 2003 backend for RAID to make it availible in XP. Tomato, (TOE)MATO

The reason why those controller cards don't to spindown is because you can't spin down drives in a RAID 0,1,5,or 6 which is what those cards support.
You should look into Unraid, which can spin down drives and be fault tolarant like RAID 5 etc.., or do like you said where you have a Spanned Dynamic Volume (Vista or above), without the fault tolerance.

[Fuzzy]

It's not even really 'hacking in the server 2003 backend', the dynamic disk system is completely in all windows OS's. All you have to do is alter the check it does to see which OS you have, so he RAID5 options can be enabled.

[mikejaner]

Originally Posted by Fuzzy

It's not even really 'hacking in the server 2003 backend', the dynamic disk system is completely in all windows OS's. All you have to do is alter the check it does to see which OS you have, so he RAID5 options can be enabled.

I stand corrected. True, the code is in the OS already. The way I should have phrased it is, "You have to hack the Windows Disk Management Subsystem DLL, and EXE etc.. to make it work"

http://www.tomshardware.com/reviews/...pen,925-2.html

[dvd_maniac]

From what I'm reading here and from personal experience raid is a questionable option for storing large amounts of video. Since speed is usually not an issue for recording/playback I decided to switch from Raid-5 to JBOD.
I had a motherboard failure with my raid-5 on it and could not recover. Luckily, I had an LTO-2 drive with backup tapes. Then I moved to a 3ware raid card and it kept dropping a couple of the discs. Once it lost 2 at the same time and I was screwed. That's when I moved away from Raid. I now use JBOD in a NAS box over the network for backup.
I just wished Sage would allow for something like shortcuts within a folder so it would scan the videos in the folder shortcut and reside them within the actual folder that Sage is using for the Import folder.

[mikejaner]

Originally Posted by dvd_maniac

From what I'm reading here and from personal experience raid is a questionable option for storing large amounts of video. Since speed is usually not an issue for recording/playback I decided to switch from Raid-5 to JBOD.
I had a motherboard failure with my raid-5 on it and could not recover. Luckily, I had an LTO-2 drive with backup tapes. Then I moved to a 3ware raid card and it kept dropping a couple of the discs. Once it lost 2 at the same time and I was screwed. That's when I moved away from Raid. I now use JBOD in a NAS box over the network for backup.
I just wished Sage would allow for something like shortcuts within a folder so it would scan the videos in the folder shortcut and reside them within the actual folder that Sage is using for the Import folder.

Sounds like you have had some bad experiences all right. JBOD doesn't do it for me, because chances are a show my wife recorded is on one of those disks. All it takes is for one show to be lost and my WAF goes down the tubes.
I have never had issues with performance on RAID with video storage. RAID5 with a motherboard controller is a bad combo, which unfortunately too many people get tricked into believing is a hardware raid. I prefer software RAID right now. I ran a software RAID5 for two years on a server 2008 box with no issues. Once I got interested in Linux RAID features and capabilities, I created a software RAID5, which over the past year has been dynamically expanded, and as we speak is currently converting to a RAID6 array on the fly. Two disks dropping at the same time is a very rare situation, but three, and somebody up above has it out for you. I did the jump from Raid5 with a hot spare, to RAID6 just to eliminate that rare situation.

[Spectrum]

Originally Posted by mikejaner

The reason why those controller cards don't to spindown is because you can't spin down drives in a RAID 0,1,5,or 6 which is what those cards support.
You should look into Unraid, which can spin down drives and be fault tolarant like RAID 5 etc.., or do like you said where you have a Spanned Dynamic Volume (Vista or above), without the fault tolerance.

True no card will selectively spin down the drives in a RAID 5/6 array but the Areca and the Adaptecs can spin down the whole array. That's something the cheaper cards just don't offer. Unraid is out. I already have 3 boxes running 24/7 and don't want to add a 4th.

Originally Posted by dvd_maniac

I had a motherboard failure with my raid-5 on it and could not recover. Luckily, I had an LTO-2 drive with backup tapes. Then I moved to a 3ware raid card and it kept dropping a couple of the discs. Once it lost 2 at the same time and I was screwed.

Ouch that sounds like bad drives, or using drives that have RAID issues like Western Digital drives with TLER turned off.

[SomeWhatLost]

Originally Posted by Spectrum

Let's beat this RAID thing to death

that seems kind of mean, possibly even a bit excessively violent... what did RAID ever due to you? why all the hate? did RAID beat you up and take your lunch money?

Originally Posted by Spectrum

Unraid is out. I already have 3 boxes running 24/7 and don't want to add a 4th.

you really should add a fourth... 3 is an odd number, a prime number, just generally a bad number... 4 on the other hand is a happy number...
or just do the whole virtualization thing...

I am currently looking for a larger storage solution... I currently have 2 infrant NV+'s with about 2T each (750G X4 raid 5/x) works well, but I am thinking a single device is better then 2... unRAID is my current favorite, although it does have a few minor drawbacks...

[Spectrum]

Originally Posted by SomeWhatLost

that seems kind of mean, possibly even a bit excessively violent... what did RAID ever due to you? why all the hate? did RAID beat you up and take your lunch money?


you really should add a fourth... 3 is an odd number, a prime number, just generally a bad number... 4 on the other hand is a happy number...
or just do the whole virtualization thing...

I am currently looking for a larger storage solution... I currently have 2 infrant NV+'s with about 2T each (750G X4 raid 5/x) works well, but I am thinking a single device is better then 2... unRAID is my current favorite, although it does have a few minor drawbacks...

Heh, I could do with missing a few lunches so maybe I wouldn't have to beat on RAID if it would come and take my lunch money

I would be all over unRaid if I didn't mind another 24/7 box or could Virtualize things on the cheap. My little gateway server is prime for combining with unRaid via esxi but I doubt the Dell p3 castoff that is being used would stand up to that very well, and I don't really want to build another box with the hardware that would be required for it. I know some of the dual core Celerons support VT-X but am not sure about using one as an esxi server.....

Ooh maybe we should go to one of the virtualization threads and wail on it for awhile

[stanger89]

Originally Posted by Spectrum

I would be all over unRaid if I didn't mind another 24/7 box or could Virtualize things on the cheap.

I had been thinking the same thing, but some research showed that you can build an unRAID box that pulls only about 25W (less drives).

[Spectrum]

Originally Posted by stanger89

I had been thinking the same thing, but some research showed that you can build an unRAID box that pulls only about 25W (less drives).

Wow that's low. Have you got a link or was it research you put together yourself?

My drives were waiting when I got home this evening and I got them all tossed in! It is kind of nice seeing 7.27 TB free on a drive

Also it's worth noting that the J-Micron sata controller on Asus boards will only work with 1 sata drive in a normal config or you can use the Asus easy raid thingamawhichy to create a raid0 or raid1 pair. Boo Hiss! I had to take my DVD drive out to put in all 4 drives lol. Soooo what I guess I'll do is get one of these and go from there.

[Bizarroterl]

Unraid doesn't require much in processing power, so almost any low power MB will do, even an ION MB. I have a Intel DQ45EK motherboard that I had a 2.8 dual core cpu. I swapped the CPU out with a 45w Celeron and saw no change in speed.