New unRaid Machine

[kevine]

Given that WHS drive extender technology is going away, I need an easy way to add new drives to my system. I chose unraid because of it's redundancy and easy of adding a new drive. It will be a separate machine from my signature machine. However, that machine is changing over to Win7 64-bit and will record SageTV, comskip, mediashrink and PlayOn. I can then finally use all of my memory too. (4gb) I will probably put 2 TB drives in motherboard hardware raid and move all the rest to the unraid server. Also adding 3 new 2 TB drives to that as well. And I plan on moving recordings over to the unRaid server using a SJQ script. BTW, anyone have one I can utilize? Any other ideas someone my have would be appreciated.

I basically have an idea to setup my new machine using the specs of the server they built for sale on their site. Only I would not buy the extra cages or expansion cards until they are needed. Any other thoughts?

Thanks.

[davephan]

I setup an unRAID server, and it is working out fine so far. If you use WD 2 TB Advanced format drives, make sure to put in the pin 7 & 8 jumper before running the preclear script and adding the drives to the array.

I have a full tower case, and I use Thermaltake iCage A2309, 3 in 3 slot cages. The drive density is lower than the 4 in 3 or 5 in 3 cages, but the ventilation is better. Here's a link to the Thermaltake iCage A2309:

http://www.amazon.com/gp/product/B00...ef=oss_product

I haven't setup SJQ yet to move recordings to the unRAID server, but I plan to do that in the future. It may take about 30 hours to run the preclear script, but you can run up to six at once.

I currently have seven 2 TB data drives, plus a 2 TB parity drive, and a 1 TB cache drive. I could fit three more of the 3 in 3 cages in the server to hold 18 drives. I don't know if I will need to expand it that far, to over 30 TB. I haven't tried recording directly on the unRAID server since I heard that the throughput may not be high enough. The throughput is fast enough for at least a couple of HD streams at once.


Dave

[KeithAbbott]

Originally Posted by davephan

I currently have seven 2 TB data drives, plus a 2 TB parity drive, and a 1 TB cache drive. I could fit three more of the 3 in 3 cages in the server to hold 18 drives.

What chassis/case are you using to support a total of 18 drives? If I understand this correctly, that's 18 external drive bays. That must be some big honkin' case...

[davephan]

Originally Posted by KeithAbbott

What chassis/case are you using to support a total of 18 drives? If I understand this correctly, that's 18 external drive bays. That must be some big honkin' case...

All the drives are internal. I am using the LIAN LI PC-G70B Black Aluminum ATX Full Tower Computer Case.

http://www.newegg.com/Product/Produc...82E16811112105

I can fit four "3 in 3" iCages along the front of the case. One "3 in 3" iCage in the lower back. One "3 in 3" iCage mounted from the top. A total of 18 internal drives could fit inside the full tower case.

I only have three "3 in 3" iCages in the case now. I would need to remove some metal floppy mount brackets to fit four "3 in 3" iCages along the front. The case has two locations for power supplies. I would need to move the power supply from the bottom rear space to the top rear space. A fifth "3 in 3" iCage could then got in the rear bottom. A sixth "3 in 3" iCage could then be hung from the top of the tower.

So, 16 data drives plus 1 parity drive plus 1 cache drive for a total of 18 drives could fit in the full tower case using "3 in 3" iCages. To expand more, higher density cages. Two of the front "3 in 3" iCages would need to be replaced by two "5 in 3" cages. Then the full tower case could be built out to a total of 22 drives. However, I don't think I'll need to go that big, since that would be getting close to 40 TB with 20 - 2 TB data drives.

Dave

[reggie14]

There are some recommended unRAID builds from the unRAID user community on their wiki. That's probably going to work better than simply trying to copy the servers LimeTech sold. You'd have a hard time getting some of the parts used in those servers.

I've been going back and forth about recommending that you buy all the hard drive cages and controllers right off the bat. One reason you might want to buy the cages right away is that the models seem to change over time and you might not be able to get matching cages in the future. I think it might be good to avoid getting any more hard drive controllers than you immediately need. Current hardware probably will never support hard drives larger than 2TB. So, getting controllers now might limit your expansion capabilities later. You might need a new motherboard to use 3TB drives in the future, but that remains to be seen. But at least you won't have a bunch of semi-useless spare hard drive controllers.

[rtrski]

Hope this isn't a hijack.

I'd looked at unRaid some time ago but never pulled the trigger; looks like they've come a long way since then. Also considered just buying the off-the-rack Drobo or Synology systems.

Based on various feedback I've soured on the Drobo a bit (seems like they may have sacrificed quite a bit of performance for 'user friendliness', and while my intent for a big disk system would be more as a backup than actual 'live' in-use media storage, I'd at least like to keep the options).

Aside from price, what do y'all see as the main benefits of unRAID vs. Synology's "hybrid RAID"? Is there a good link you could suggest for learning the pros/cons in terms of backup integrity and/or performance? I kind of doubt right now I'd get beyond the need for about 6 TB actual storage (which I could achieve with 5 2TB drives in dual redundancy mode). At least, not anytime soon....

[reggie14]

I'm an unRAID user, and I've never used a Synology system. So, take this with a grain of salt.

Synology hybrid raid systems should be quite a bit faster than an unRAID system, particularly for writes. It looks like Synology hybrid RAID still stripes writes across drives. Allegedly, there's only a 10% performance hit compared to RAID5. Writes on unRAID systems, however, top out around 35-40MB/sec.

However, except for the performance hit, there are some nice benefits to not striping writes across drives. First, not all drives need to be spun up all the time- only the drive you're reading from (or writing to, plus the cache drive). That saves on power, reduces heat, and might help extend the lifetime of the drives (or might hurt it, I think the jury is still out on that one). Second, each data drive in an unRAID system is just a reiserfs-formatted drive. You could pull out a data drive and mount it in any linux system to get the data off of it (potentially useful if your unRAID server goes down, or if you have multiple hard drive failures).

I know unRAID can only recover from a single disk failure. I'm pretty sure that's true of Synology Hybrid RAID too.

[kevine]

Originally Posted by davephan

A sixth "3 in 3" iCage could then be hung from the top of the tower.

That's awesome! Very inventive.

I really like your setup. How loud are the 3x3 fans? An advantage of the Icy Doc (besides 5x3) is quiet fans.

[kevine]

Originally Posted by reggie14

I've been going back and forth about recommending that you buy all the hard drive cages and controllers right off the bat. One reason you might want to buy the cages right away is that the models seem to change over time and you might not be able to get matching cages in the future. I think it might be good to avoid getting any more hard drive controllers than you immediately need. Current hardware probably will never support hard drives larger than 2TB. So, getting controllers now might limit your expansion capabilities later. You might need a new motherboard to use 3TB drives in the future, but that remains to be seen. But at least you won't have a bunch of semi-useless spare hard drive controllers.

Great suggestions. Whenever I hear suggestions about future expansion, I always think to myself "You'll never need that". But we all know reality comes to bite us later on. I remember when I said "A 20 MB Hard drive! What am I gonna do with all that space?" Now I can get 1000x that in a device smaller than my pinky finger. lol

[kevine]

Originally Posted by rtrski

I kind of doubt right now I'd get beyond the need for about 6 TB actual storage (which I could achieve with 5 2TB drives in dual redundancy mode). At least, not anytime soon....

See my post below. I gotta laugh again. lol

[rtrski]

Yeah, I was just marvelling with my dad that my entire music collection, some 400 CDs or so (never did the iTunes thing), at 192KBPS WMA, fits on a single 32 GB USB drive plugged into a jack in my new car (2011 Kia Sportage). And to think I used to be impressed by trunk-mount 6-disk CD changers!

I've currently got 3 1TB drives in my Sage server, and am fairly slowly amassing movies and the like. About 1 TB of current movie storage, 1 TB current recorded TV, and 1 TB spare still. As the 1TB TV is mostly a 'revolving door' its pretty safe to assume that the stuff I'm keeping long term (either movies or TV episodes pulled from DVDs after compilation at end of season - I hate all the crappy network overlays they're doing these days and don't want to keep those copies forever) won't add up to 4 TB too soon.

Of course, I haven't taken the BD leap yet, either.

Bottom line, while I might want way more than 6 TB someday "soon" (say 3-5 years window) any disk backup solution I build probably doesn't have much more than that sort of viability window anyway. Gotta keep that consumer economy moving by upgrading every few years, after all just bought an HD300 to replace my 200 when I upgraded to Sage7, right? :P

[davephan]

Originally Posted by reggie14

Second, each data drive in an unRAID system is just a reiserfs-formatted drive. You could pull out a data drive and mount it in any linux system to get the data off of it (potentially useful if your unRAID server goes down, or if you have multiple hard drive failures).

I tried taking one of my drives out, and I could not read it with a Ubuntu system. However, this was before I learned that I had a WD 2 TB Advanced Format problem. I've since fixed the problem by copying all the data off the system, rebuilding the system with the pin 7 & 8 jumped, and then re-copying all the data. I may try hooking up one of the data drives again to a Ubuntu system to verify that the files can be read on a Linux system.

Originally Posted by kevine

How loud are the 3x3 fans? An advantage of the Icy Doc (besides 5x3) is quiet fans.

The fans in my "3 x 3" iCages aren't very loud. I have the server in the basement, which is away from the normal 'living' space. I could have the server in an occupied room if the basement wasn't available, since all the fans in that server aren't very loud.

Dave

[reggie14]

Originally Posted by kevine

Great suggestions. Whenever I hear suggestions about future expansion, I always think to myself "You'll never need that". But we all know reality comes to bite us later on. I remember when I said "A 20 MB Hard drive! What am I gonna do with all that space?" Now I can get 1000x that in a device smaller than my pinky finger. lol

And the more storage space you have the faster you'll accumulate data. I set up my unRAID server sometime around April with 2x2TB data drives and a 2TB parity drive. I thought 4TB would last a while. Well, since then I've bought 3 more 2TB drives. My server is no where near full, but I didn't expect to be using 4.5TB at this point.

Now I'm really hoping that the last 5.5TB of space lasts me a while. I didn't know about the <3TB limitation at that point, or the problems that unRAID has with advanced format drives. It probably wouldn't have changed anything for me, since there were a number of reasons I went with unRAID over a specialized NAS device, but it was a bit of a surprise. Now I'm trying to avoid buying any new hardware until I know Lime Tech is going to do something about those issues (which will likely require new hardware to come onto the market).

[reggie14]

Originally Posted by davephan

I tried taking one of my drives out, and I could not read it with a Ubuntu system. However, this was before I learned that I had a WD 2 TB Advanced Format problem. I've since fixed the problem by copying all the data off the system, rebuilding the system with the pin 7 & 8 jumped, and then re-copying all the data. I may try hooking up one of the data drives again to a Ubuntu system to verify that the files can be read on a Linux system.

Having the drive misaligned shouldn't impact your ability to mount the drive. It just should have impacted performance, since, as I understand it, when doing writes the hard drive will essentially have to access two sectors for every sector written, instead of just one.

There's an FAQ entry that discusses mounting the data drives, but its light on details on how to do it. I'm sure people over on the LimeTech forums would be happy to help you out if you run into problems next time you try to do it.

[Bizarroterl]

Originally Posted by reggie14

Writes on unRAID systems, however, top out around 35-40MB/sec.

You can improve write speeds significantly with a cache drive. For the insane you can use a SSD for the cache. Then 75-85MB/s is possible (assuming your source can keep up).

[stanger89]

Originally Posted by reggie14

Having the drive misaligned shouldn't impact your ability to mount the drive. It just should have impacted performance, since, as I understand it, when doing writes the hard drive will essentially have to access two sectors for every sector written, instead of just one.

Yeah, I think mine mounted fine in linux (kubuntu live cd), though I had to do it manually, they weren't auto-mounted.

FWIW, I'm using a Supermicro X7SPA-HF for my unRAID box. I haven't measured the power but unRAID's testing had it under 30W idle, 50W idle when loaded with a Supermicro SATA card and 12 drives (IIRC).

[RobJ]

Is there a good link you could suggest for learning the pros/cons in terms of backup integrity and/or performance?

Reggie14 responded with an excellent synopsis of the advantages of a non-striped solution like unRAID. If you want more info and pros and cons, see General Questions about unRAID. While we are unRAID users, we tried to be fair. Unfortunately, some wiki info is becoming increasingly dated.