Storage questions, NAS, WOL, lots of stuff!

[sunray]

Originally Posted by Kirby

OK, so it looks like if I want a large drive array I need to just make my own, similar the the unRaid system. I have enough components for a PC, buy a big case like the Cooler Master case they show, or other similar, and I could even go about builing my own linux install onto a USB flash drive like they do it.

But things like ReadyNAS just seem to limited, only 4 drives max. I want to be able to put in say up to 12-15 drives, and be able to swap them out for larger and larger drives as they come out and as space requirements dictate.

I am aware of the firewire speed vs. USB 2 speed, and that external drives are hotter, but surprisingly the WD My Book 500GB drive I have runs rather cool, even when its reading/writing 4 HD streams at a time. A fan would definitely help, but for recording and playback, 4 HD streams over USB 2.0 works great, whereas firewire (400 or 800), or eSATA is going to require a PCI card of which I dont have an empty slot on my sage server. Now if I can find a PCI-e X1 version of a firewire card, that would be a possibility.

If you want a case with room for a lot of drives the CoolerMaster Stacker is a good choice. If you use IStar iStorm 7 3 into 2 bay hard drive adaptors instead of CoolerMaster's 4 into 3 bay hard drive adaptors you can fit 15 hard drives plus a optical drive into the Stacker case. I'm doing this in my media server. Depending on which version of the case you buy it comes with space for 1 or 2 PSUs.

The MyBook case may feel cool but that doesnt mean that the drive inside is running cool, plastic cases like the MyBook tend not to transmit heat well which is why most of the more expensive external cases use metal. Have you checked the temp with a temp probe? I dont know of any software that will allow hd temps to be checked for a USB2 or Firewire drive.

As far as the ReadyNAS is concerned you can add more of them if you need more storage. For about the same amount of money as a ReadyNAS there is also the Buffalo TeraStation. Another way to go which is cheaper is to buy one of the low cost Linux based NASes like the Linksys NSLU2, NetGear SC101, DLink DSM-G600. The Linksys uses external USB2 drives and has a very active developer community. From the reviews I've read the low cost NASes tend to be slow when writing files though.

[Kirby]

I think I will stick with my Thermaltake Armor case, as I can get 18 drives in with the use of 2 '4 in 3' and 2 '3 in 2' adapters, as well as the internal mount for 3 drives, and 1 drive located in the power switch bay.

Now I just need to figure out which RAID controllers to get.

[BobPhoenix]

Originally Posted by Kirby

I think I will stick with my Thermaltake Armor case, as I can get 18 drives in with the use of 2 '4 in 3' and 2 '3 in 2' adapters, as well as the internal mount for 3 drives, and 1 drive located in the power switch bay.

Now I just need to figure out which RAID controllers to get.

I've got a Lian Li V2000 case with 3 of the 3 in 2 adapters to bring my HD total to 21 drives currently with one CD and no floppy. In the summer it gets too hot to leave it on in the basement so I shut it down for half of the day to keep the heat in the basement down.

[Kirby]

Originally Posted by BobPhoenix

I've got a Lian Li V2000 case with 3 of the 3 in 2 adapters to bring my HD total to 21 drives currently with one CD and no floppy. In the summer it gets too hot to leave it on in the basement so I shut it down for half of the day to keep the heat in the basement down.

So its a good alternative heat source for the cold Michigan winters, thanks for the tip!

That brings me to the next question, is that your sage server or a storage server? If storage, anyone know how long it can be offline before (if ever) Sage will eliminate associated Guide data for those shows?

[BobPhoenix]

Originally Posted by Kirby

So its a good alternative heat source for the cold Michigan winters, thanks for the tip!

That brings me to the next question, is that your sage server or a storage server? If storage, anyone know how long it can be offline before (if ever) Sage will eliminate associated Guide data for those shows?

It's a storage server and the Guide data is long gone for it's files. I should have xml files for all of them (I hope) and I'm going to write something to display those xml files someday. So I'm afraid I don't know for sure but I believe that as long as the drive is not accessable to Sage it will keep them in the list until it can actually verify that they don't exist any more.

[Kirby]

If thats how Sage works thats great news. My big fear is that my storage server will somehow power down and sage will stay online, and a few days go by without me trying to access a recording from that storage server, and then find I have terabytes of recordings with no info again.

[PGPfan]

Originally Posted by Kirby

OK, so it looks like if I want a large drive array I need to just make my own, similar the the unRaid system. I have enough components for a PC, buy a big case like the Cooler Master case they show, or other similar, and I could even go about builing my own linux install onto a USB flash drive like they do it.

But things like ReadyNAS just seem to limited, only 4 drives max. I want to be able to put in say up to 12-15 drives, and be able to swap them out for larger and larger drives as they come out and as space requirements dictate.

I am aware of the firewire speed vs. USB 2 speed, and that external drives are hotter, but surprisingly the WD My Book 500GB drive I have runs rather cool, even when its reading/writing 4 HD streams at a time. A fan would definitely help, but for recording and playback, 4 HD streams over USB 2.0 works great, whereas firewire (400 or 800), or eSATA is going to require a PCI card of which I dont have an empty slot on my sage server. Now if I can find a PCI-e X1 version of a firewire card, that would be a possibility.

Hi Kirby,

I don't know how much exploring of the unRaid site you did, but you can (and should, IMHO ) get just their flash drive. It's the software that makes the unRaid special, and that is what you are paying for IMHO. You have talked about drive power management alot in this thread, that is specifically were unRaid REALLY shines over and above other options. Tom (the unRaid dev) has a lot of proprietary code in his flash to accomplish this. He is currently working on spindown support of SATA drives, but right now this is only available on a PATA system. All I bought was the USB flash from him and did the rest myself. I've got a 1.5 TB system currently with a TOTAL cost of about $675, and it is fully fault tolerant.

Not bad at all for the money, and you can add up to 12 drives replacing smaller ones with larger ones as you find them on sale, etc.

-PGPfan

[BobPhoenix]

Originally Posted by Kirby

If thats how Sage works thats great news. My big fear is that my storage server will somehow power down and sage will stay online, and a few days go by without me trying to access a recording from that storage server, and then find I have terabytes of recordings with no info again.

As a safety net you could install this plugin and save the XML output somewhere. Then if NIELM or someone else writes a way to put the data back into the SageTV database you would be able to get it back. And until you get an answer from someone that knows for sure I would just leave it on. The only reason I turn mine off is because of heat. During the Winter I use the Storage Server to ReEncode my videos into MPEG-4 format so it is on 24/7 on a UPS.

[stanger89]

Originally Posted by Kirby

OK, so it looks like if I want a large drive array I need to just make my own, similar the the unRaid system. I have enough components for a PC, buy a big case like the Cooler Master case they show, or other similar, and I could even go about builing my own linux install onto a USB flash drive like they do it.

Couple comments from experience

I've got a 1.75TB array on a 3ware 7506-8 (8 drives), as well as a ReadyNAS X6 (though I'd get the NV if I were buying now)

First anything much beyond 8 drives (even 8 drives to a degree) gets increasingly impractical. I've had some trouble powering 8 drives (startup only). I really wouldn't look forward to managing something bigger, it just becomes a logistical nightmare.

Second, with the size you can buy HDDs today, it seems less and less necessary to go with vast numbers of HDDs.

I have my SageTV/JRMC/storage server all combined in one unit, started with the SageTV server, grew my storage until I had 10 drives total. This works pretty well, just adding an array to an existing PC/server, but when I went to investigate means to add more space, it became clear just how impractical it would be to add more.

Adding another 8 drives (my initial plan) would have involved:
New RAID card (probably a 3ware 9500s)
Many drives
plus one of the following
A new case with new PSU to power 16 drives (I'd probably want something like 1KW for that)
or
An external SATA HDD enclosure (ref macgurus)

If I were looking at adding storage to my network, IMO, NAS is the way to go, especially with the proliferation of "cheap" NAS options.

Plus there are good reasons to segregate data, and if you start partitioning stuff off, the need for a huge monolithic array drops quickly.

But things like ReadyNAS just seem to limited, only 4 drives max. I want to be able to put in say up to 12-15 drives, and be able to swap them out for larger and larger drives as they come out and as space requirements dictate.

Is it really that limiting though? With 4 drives you can easilly get 2TB, with redundancy. That's about 400 DVDs (full quality, no recompression) or over 200 hours of HD (maybe more, lots of channels run in the 6GB/hr range). For 15 drives, you're talking 10TB, do you really need that now, and when you do, will it take more than 4 drives to achieve it?

To expand on non-monolithic comment, what are you going to put on your server? DVDs, recordings? If so, do you have more than 400DVDs? Do you have more than 200hrs of HD recordings that you want to save?

Seems like even if you have several hundred DVDs (like me) and many HD recordings, it wouldn't be bad to have them separated.

Consider the option of two ReadyNAS NVs, you could have 4TB of redundant storage, and nice organization of the two, and if you lose one, you're not out anything. Plus the failure risk of 1/4 is much, much lower than 1/15.

FWIW, at first, I too was dissapointed that there wasn't a 6-8 drive ReadyNAS, but as I've looked at it more, especially with 750's available, I just don't see the need for more than 4 drives in a given array.

Oh, and FWIW, I'm definitely not a fan of the unRAID, the idea of 8+ more shares/mapped drives on my network is maddening, plus having to manage data across 8 or more volumes.

I am aware of the firewire speed vs. USB 2 speed, and that external drives are hotter, but surprisingly the WD My Book 500GB drive I have runs rather cool, even when its reading/writing 4 HD streams at a time. A fan would definitely help, but for recording and playback, 4 HD streams over USB 2.0 works great, whereas firewire (400 or 800), or eSATA is going to require a PCI card of which I dont have an empty slot on my sage server. Now if I can find a PCI-e X1 version of a firewire card, that would be a possibility.

Personally if I'm going to have a separate box lying around, I'd much prefer it to be autonomous than tied to a particular PC, ie I'd much rather have a NAS than an "external" HDD. That said, Norco makes an eSATA RAID external enclosure, that's not too bad, if you can manage RAID-5 it might be worth a look.

[stanger89]

Oh, and there's a LOT of stuff the ReadyNAS can do beyond just providing storage, stuff I'm aware of....

Print Server
FTP server

But the really cool thing I just started using (right now) is the backup capabilities. I just set it up to "log on" to my SageTV server and make a backup copy of the Sage directory every day.

[Kirby]

I've been reading the ReadyNas forums, and I really dont know if I follow or not. On one hand it seems 750GB drives are supported, but on another hand it sounds like they are not (and I'm talking with the beta firmware). I understand that 4@750gb gives a RAID5 just over 2TB, and it would waste about 50GB or so (which could be made into another volume right?).

If I am understanding the whole X-RAID thing properly, I could buy a 1 or 2 drive unit, and expand at my leisure until I get 4 drives in it. At the point that it has 4 drives, it essentially is a RAID-5 system correct? 1 drive is just the drive, 2 is RAID 1, 3 or 4 drives become RAID-5?

My thoughts at this point, at least so far as storage is concerned, is to do a ReadyNAS NV and load it up with the largest of my SATA drives (400's) and also look at running that NASLite V2 on my existing storage server with all my PATA drives. I see no sense in throwing that system away completely, I could get a card like the Highpoint Rocketraid 464, which can handle 8 IDE drives (2 RAID5 arrays) and supports at least 400GB drives (which all my PATA drives are that or below). Seems like a decent setup, and shouldnt be too demanding on my 750 watt PSU (its already running 8 drives plus an OS drive).

Then going forward I can just add in more ReadyNAS type setups.

[BobPhoenix]

Originally Posted by Kirby

I've been reading the ReadyNas forums, and I really dont know if I follow or not. On one hand it seems 750GB drives are supported, but on another hand it sounds like they are not (and I'm talking with the beta firmware). I understand that 4@750gb gives a RAID5 just over 2TB, and it would waste about 50GB or so (which could be made into another volume right?).

If I am understanding the whole X-RAID thing properly, I could buy a 1 or 2 drive unit, and expand at my leisure until I get 4 drives in it. At the point that it has 4 drives, it essentially is a RAID-5 system correct? 1 drive is just the drive, 2 is RAID 1, 3 or 4 drives become RAID-5?

My thoughts at this point, at least so far as storage is concerned, is to do a ReadyNAS NV and load it up with the largest of my SATA drives (400's) and also look at running that NASLite V2 on my existing storage server with all my PATA drives. I see no sense in throwing that system away completely, I could get a card like the Highpoint Rocketraid 464, which can handle 8 IDE drives (2 RAID5 arrays) and supports at least 400GB drives (which all my PATA drives are that or below). Seems like a decent setup, and shouldnt be too demanding on my 750 watt PSU (its already running 8 drives plus an OS drive).

Then going forward I can just add in more ReadyNAS type setups.

I don't recommend this but I have 21 drives running on a 600W. Uses only 450W according to UPS.

[stanger89]

Originally Posted by Kirby

I've been reading the ReadyNas forums, and I really dont know if I follow or not. On one hand it seems 750GB drives are supported, but on another hand it sounds like they are not (and I'm talking with the beta firmware). I understand that 4@750gb gives a RAID5 just over 2TB, and it would waste about 50GB or so (which could be made into another volume right?).

Last "official" word I saw over there was that using the beta firmware, 750s were supported, but yes, you lose that 50GB. Not sure if that can be a separate volume in an X-RAID setup or not.

If I am understanding the whole X-RAID thing properly, I could buy a 1 or 2 drive unit, and expand at my leisure until I get 4 drives in it. At the point that it has 4 drives, it essentially is a RAID-5 system correct? 1 drive is just the drive, 2 is RAID 1, 3 or 4 drives become RAID-5?

If I understand correctly, it's actually more like RAID-4, where there's a drive dedicated for parity, vs being spread around the array.

[Kirby]

Ok that makes sense about the RAID 4.

After doing some quick cost calculations, it seems a ReadyNAS NV with 4 drives is best done with 500GB drives, resulting in a total cost of about $0.95 per GB for a 1.5TB array. Expensive compared to the $0.38/GB price of the drives, but 750GB (and hopefully larger) drives will be able to be used fully down the road.

Of course the other way to look at it, you could just buy double the number of drives you need, "mirror" them once the first drive is full, and take the mirror offline. Then if that drive ever failed, you would have an offline backup, total cost about $0.78 per GB. Load up on some removeable drive trays and no need for RAID 4/5.

[Berg]

Originally Posted by Kirby

Of course the other way to look at it, you could just buy double the number of drives you need, "mirror" them once the first drive is full, and take the mirror offline. Then if that drive ever failed, you would have an offline backup, total cost about $0.78 per GB. Load up on some removeable drive trays and no need for RAID 4/5.

If I may chime in ... as another NAS Lite user and one who also has an extensive "digital media library", I have had the opportunity to think and discuss at lenght to RAID or not to RAID, and I have concluded not to RAID.

Aside from the cost argument, RAID does nothing to protect you from a data loss because of fire/flood/theft/etc., so to really back up your data, you really need some form of off-site backup. I have also read many accounts of a RAID recovery not working out on the type of RAID hardware you and I can afford, not to mention, multiple drive failures affecting one server (multiple drive from the same "batch" failing at around the same time) making a typical RAID setup useless.

Today, I use two NAS Lite servers to store my music and "personal" digital media. One sits at home while the second one replicates my main server over RSYNC at a buddy's house. The cost of doing the same for DVDs is too high for my taste today, but that is quickly changing. I may very well move over to this type of setup in the fall when the drive price will apparently be significantly lower.

My two cents Canadian .... which is also worth quite a bit more than a year or two ago

[Kirby]

Originally Posted by Berg

My two cents Canadian .... which is also worth quite a bit more than a year or two ago

Booooooooooooooooooo!!!! Made that future Expressvu subscription more expensive!

[Berg]

Originally Posted by Kirby

Booooooooooooooooooo!!!! Made that future Expressvu subscription more expensive!

Isn't it funny ... the Canadians want the US channels and the Americans want the Canadian channels.

Crazy world, isn't it ?