Using Microsoft Distributed File System (DFS)??

[steingra]

The Distributed File System (DFS) allows files and directories in various places to be combined into one directory tree. I am wondering if anyone has used this in conjunction with their SageTV setup. Here is a google search

http://www.google.com/search?hl=en&q...er&btnG=Search

I am trying to find a way to put a bunch of physical drives into one logical shared drive. So far DFS seems like it might work. But maybe not best solution.

I already have 5TB of drives running. And with 1TB drives coming down in price I imagine I will have more. But I hate to run so many file servers to hold all the drives. So I was thinking if I could shove more of them into one large server case, and run something like DFS...then I could share all the drives as \\ServerName\VideoStorage and turn off some of the other Windows machines. I know there are things like Drobo, etc...but they are either firewire or USB2...and I dont think that would provide enough bandwidth for me. Maybe for simple recording SD shows. I currently only use two tuners on sagetv server. But I am planning on getting an HDHomeRun and using that to record some stuff in HD from my OTA setup. So thats going to put more pressure on the network and sotage systems.

I dont care about fault tolerance either. If a drive dies...oh well they are just tv shows or movies,,,I dont care that much if I lose them. But I dont want to lose the entire volume if one drive dies.

****************************
Dynamic Disks Windows 2000
I also remembered using dynamic disks in Windows 2000. But I got hosed by that big time since I didnt understand it very well and the whole thing crashed on me and I lost all the data. It was just videos and shows....but still. ugh

Here is a link
http://www.theeldergeek.com/hard_drives_10.htm

I see they are talking about creating Spanned Drives for a volume. And you can add in up to 32 drives. So I suppose that would solve this problem for me. But the guy gives a warning about using them. And I remember I suffered from Windows Dynamic disks a few years ago.

But I do like the concept of having a whole bunch of drives all shared as one volume, all shoved into one big server case. I need it to be reliable though.

*********************************

Linux Logical Volume Manager???

Link....http://www.softpanorama.org/Commerci..._manager.shtml

I wonder how this might work in Linux? Anyone work with the LVM in linux?

Seems like an aweful lot of information to just share a bunch of drives.

I have worked with Linux in the past, but am no guru with it. hmmmm

[GKusnick]

DFS doesn't aggregate physical drives into a single large logical partition. What it does is to make network shares on different servers appear as folders under a single logical share. You can specify more than one target for a DFS link, but that doesn't pool the storage from those targets. Multiple targets are assumed to be mirrors of each other; when you refer to a link, the DFS client selects one of the targets (generally the one topologically nearest the client) and ignores the rest. So I don't see how DFS is going to solve your problem.

You might take a look at Windows Home Server, which is probably closer to what you're trying to achieve.

[CollinR]

Just buy an old decommissioned SCSI storage box. Sometimes they will pay you to come get it.

[stanger89]

I was going to say eSATA, of course that's solving a different issue than creating one logical volume.

[bcjenkins]

Starting with this post: http://forums.sagetv.com/forums/show...8&postcount=29

Read the thread. Excellent info on linux and performance tuning.

B

[bluenote]

I know it's handy to have just one or a few sage directory's but...

why not just shove them all in one box. if they are big enough, share them out on by one. if some of the drives are too small, do a regular SPAN under windows to make aggregate drives, but not all of them together. (yes i've been bit on dynamic disks too.. %$!$%!).

Then, just configure sage with all those UNC shares.

solved, nice and simple. just not dead tidy, but who cares?

why are you using multiple machines to do this anyways? not enough controllers in one machine?

[steingra]

Originally Posted by cnovak

why are you using multiple machines to do this anyways? not enough controllers in one machine?

Lets see if I can explain in a little more detail.

Basically there are too many hard drives to fit in a physical space in the cases that I have, or I run out of IDE ports. Currently, I have about 4.5 TB of storage. I want 10TB or so. The average drive size is 250GB in the pc's.

If I knew for sure I could use external USB 2.0 drives reliably, I could just buy a bunch of USB hard drive enclosures, and stack them up, plug them in use them as shared folders on one Windows machine. I just have not taken the time to find out if SageTV will work OK or not with external USB drives. I would only be recording one SD show at a time...and possibly watching one too at the same time once in a while.

Although in the near furture, I am going to get an HDHomeRun. And I know that will use more bandwidth and disk i/o vs doing just SD shows. So maybe recording just 1 SD show + watching 1 SD show would be fine on external USB drive. The HDHomeRun might be too much (for the usb drives) if added as a recording source.

I have noticed a problem in the past after hooking up several USB devices and hard drives to a USB hub. It looked like too many hardware interrupts were happening on the laptop, which spiked the CPU to like 40%, and I wasnt even doing anything. As soon as I unplugged the USB devices, back down to 1 or 2%...soooooooooooooooo anyway. Not sure if USB will work for what I would like it to. But that would solve one of my problems (having too many pc's lying around) if I could just plug in a bunch of external drives and stack them up.

At this point, I am assuming a USB external drive consumes a lot less power than a PC does. Plus they dont need a Windows OS to run. Which is less $$$ to spend. And it means I dont have to maintain an OS either. I am sure I could run a Linux OS, and hook up the USB drives to it. But questions still linger.....will USB 2.0 hold up to my needs??? Can USB do it?

This only solves one of my problems...which is to get rid of some PC's while still having lots of space.

The other issue is what I previously mentioned, I would like to have everything in one recording volume like \\ServerResource\Videos

Because of nasty problems with Dynamic disks I lost a bunch of drives and shows. So this had made me rethink my storage solution backend.

I have just been trying to figure out the best, cheapest, easiest to maintain method to save a LOT of shows and avoid some of the problems I have had because of hardware failures and whatnot.

It seems like if I can have just one Volume with say 20+ drives in it, and sageTV just sees it as one big drive...then I have very few problems to worry about. If one of the drives dies...oh well, stick a new on in the USB enclosure, and add it back into the volume (assuming they are just hot-pluggable). I would much rather lose a single drive, than an entire machine again.

Plus, 20+ USB drives should use a lot less power then 5 pc's with the drives in them, and they will take up less shelf space.

Im thinking about just trying several experiments over the winter to see what shakes out.

[stanger89]

Like I said, I'd look at eSATA or SAS (though SAS isn't exactly cheap). All the benefits of external USB (that being it's external and lower power than a PC) but it's a real SATA link so you get full SATA performance, just like it's in the box.

[flavius]

I'm just wondering how much POSIX compliance is in Vista

[mayamaniac]

Originally Posted by steingra

The other issue is what I previously mentioned, I would like to have everything in one recording volume like \\ServerResource\Videos

With the way SageTV handles disk space, I don't see the need to have one volume, unless you need it for other reasons and not just SageTV.

Because of nasty problems with Dynamic disks I lost a bunch of drives and shows. So this had made me rethink my storage solution backend.

Only use Dynamic Disk if you have a backup system, its like software Raid 0, unless you don't really care about losing the data.

I have just been trying to figure out the best, cheapest, easiest to maintain method to save a LOT of shows and avoid some of the problems I have had because of hardware failures and whatnot.

The cheapest way in my opinion is to replace those smaller drives with 500G SATA drives and up. Instead of having 20+ drives, you might get away with just 10, that saves you a lot of headaches there in terms of energy cost, server space, having enough IDE/SATA interfaces, hardware failures (more drives, more chances of failure), etc.

It seems like if I can have just one Volume with say 20+ drives in it, and sageTV just sees it as one big drive...then I have very few problems to worry about.

Again, with the way SageTV handles disk space, if this is strictly for SageTV use, I don't see the need to have just one volume.

[stanger89]

Originally Posted by mayamaniac

The cheapest way in my opinion is to replace those smaller drives with 500G SATA drives and up. Instead of having 20+ drives, you might get away with just 10, that saves you a lot of headaches there in terms of energy cost, server space, having enough IDE/SATA interfaces, hardware failures (more drives, more chances of failure), etc.

That's right where I'm at today, I've got a bit over 3TB of space (not including my Sage recording drives) spread over 12 HDD, 8 250s and 4 500s. I could basically replace all that with just 4, 1TB drives, and that would save me 80W.

Again, with the way SageTV handles disk space, if this is strictly for SageTV use, I don't see the need to have just one volume.

When drives were smaller I think there was more of an argument for the convenience factor of having a single volume.

[mayamaniac]

Originally Posted by stanger89

That's right where I'm at today, I've got a bit over 3TB of space (not including my Sage recording drives) spread over 12 HDD, 8 250s and 4 500s. I could basically replace all that with just 4, 1TB drives, and that would save me 80W.

I have about 4.5TB right now, about 11 drives. I have an OS drive, 7 500Gs, and 3 250Gs. If I plan to add more space, I would start by replacing the 250s with the biggest drive I can afford.

When drives were smaller I think there was more of an argument for the convenience factor of having a single volume.

He said his drive are 250Gs on average, they are not small, he just has a lot of them. Besides the initial first setup of setting the paths in SageTV, there's not much more you have to do, SageTV takes care of the rest. Granted at some point, you run out of drive letters though.

[stanger89]

Originally Posted by mayamaniac

I have about 4.5TB right now, about 11 drives. I have an OS drive, 7 500Gs, and 3 250Gs. If I plan to add more space, I would start by replacing the 250s with the biggest drive I can afford.

Yeah, that's my plan. For a while I was hooked on the "buy the best $/GB" bandwagon, and it seems to make sense, you get the most space for your $ input. But that's only in the short term. In the long run, you end up running out of space sooner, replacing drives sooner, and spending more in electricity running them. Let me clarify the "running out of space" part, what I mean is if you go with small drives, you fill each one up faster, so you run out of bays in your case, ports on your controller, power capability on your PSU sooner. Basically, the way I look at it today, $/GB is good up front, but you need to upgrade sooner. Kind of like buying "bargain" PC parts or "cutting edge" ones. I buy the most horsepower I can because it means upgrading less.

Some perspective, 500GB drives, the $/GB leader today, run about $100, or about $0.20/GB. In comparison, 1TB drives run $300, or $0.30/GB.

The initial reaction is "That's 3 times as much!" however it's also twice the storage, so it's only 50% more expensive. Further, if you go with one of the WD Caviar GPs, they use 1/2 the power of other drives, so they're actually 4x more efficient (W/GB), power wise, than a 500GB drive.

He said his drive are 250Gs on average, they are not small,

For me they are

...he just has a lot of them. Besides the initial first setup of setting the paths in SageTV, there's not much more you have to do, SageTV takes care of the rest. Granted at some point, you run out of drive letters though.

From the SageTV side, I agree, but I look at it from the management perspective. I only record to two drives because I never save anything. But what I do do is put my DVDs on my servers for availability throughout the house. With my current setup my ~400 DVDs are spread out over only two shares (one on my NAS, one on my server). That's not too bad. But I'd hate to imagine dealing with them if all the drives were separate, managing space across 12 shares is a little ridiculous.