NAS Throughput Question

[Media Quest]

I currently run a 2Tb and a 1.5TB HD in my Pentium 3 running windows 7 and Sage Media Server. I have 3 HD 200s, and it works fine.

I am looking for more capacity and redundancy, as well as file sharing and backup functions.

I run it and several computers through a 100mb unmanaged Switch

I am considering one of the Synology products, the DS410, DS411+, and DS411J

or perhaps one of the QNAP products.

My question is this: How much throughput capacity do I need? I suppose the worst case scenario is that I might record multiple programs while playing back multiple programs.

It also seems like performance can be improved by recording locally on the server and then moving the finished recordings to the NAS?

Without an unlimited budget, I would love to have some feedback on how much network capacity recording and playing back HD video consumes so I can choose the best NAS box for my needs.

Thanks,

[Fuzzy]

Personally, I don't like the idea of Sagetv + a NAS. I know there are plenty here that are doing so, but I'd like to think most of those already had the NAS before going with sage. To me, if you've already got a 24/7 available server (the SageTV Server) on your network, the NAS is just added cost/complexity. If you want more storage, add it to the sage server. There are two situations to look at. First - watching livetv. This involves recording from your tuner on the sagetv server, sending that stream to the NAS, reading the file BACK from the NAS to the server, then streaming it to the Extender. Some of this is mitigated by recording to the local storage, and then using some script to move recordings to the NAS - but even then, to play on the extenders, you'll end up streaming from NAS -> Server -> HD300. With clients, it's a little better, as they can read from the NAS directly.

For the cost of most NAS products, you could buy a really nice, high storage count case, and a decent PCIe SATA card, and just add drives to your server. Sage works best with MANY local recording drives, as it will spread the throughput among the drives (when set to Bandwidth mode).

[GKusnick]

Originally Posted by Media Quest

I run it and several computers through a 100mb unmanaged Switch

If you're serious about recording directly to NAS you will probably want to upgrade to gigabit LAN.

[Nelbert]

Originally Posted by Media Quest

Without an unlimited budget, I would love to have some feedback on how much network capacity recording and playing back HD video consumes so I can choose the best NAS box for my needs.

Thanks,

Without knowing where you are it's difficult to say as it varies between broadcasters.

In the UK, freesat are broadcasting hd around the 8-10 Mbit/s rate iirc, so the same as DVD maximum bit rates, with SD channels ~2-4Mbit/s. DVB-T rates are a little lower. By contrast, the Russian HD channels we watch are usually closer to the 15 Mbit/s rate region if not higher.

This is obviously reflected in the size of the recording, not to mention visible quality of the image as well.

[sflamm]

For the cost of most NAS products, you could buy a really nice, high storage count case, and a decent PCIe SATA card, and just add drives to your server. Sage works best with MANY local recording drives, as it will spread the throughput among the drives (when set to Bandwidth mode).

I use a very similar approach. Buy a 4-in-3 internal hot swap enclosure and a Raid card (i.e. Adaptec 3405) and run Raid 5. Make sure you by hard drives that support TLER for max reliability(but not required). Cheaper overall cost and higher performance since it is local storage.

Actually I run multiple hot swap Raid Arrays with multiple Raid Adapter cards
I/O is > 200MB/s sustained. Burst is close to 400MB/s.

[stanger89]

Originally Posted by Media Quest

Without an unlimited budget, I would love to have some feedback on how much network capacity recording and playing back HD video consumes so I can choose the best NAS box for my needs.

If you're going to record to it, you're going to want something rather high performance, something probably along the lines of a ReadyNAS Pro, or NVX (or whatever their latest high performance option is). Basically figure this:

Blu-ray: 50Mbps
OTA HD: 20Mbps
HD PVR: 13Mbps

The thing to remember with recording is that you need pretty good random I/O performance, so you'll probably want something rated or benched a lot higher than just those numbers would say.

Originally Posted by Fuzzy

Personally, I don't like the idea of Sagetv + a NAS. I know there are plenty here that are doing so, but I'd like to think most of those already had the NAS before going with sage. To me, if you've already got a 24/7 available server (the SageTV Server) on your network, the NAS is just added cost/complexity. If you want more storage, add it to the sage server.

The problem is there aren't any really "good" options for redundancy in windows. If you go the RAID route like sflamm, you can very quickly eclipse the cost of an external NAS. That's how I ended up with one in the first place.

I personally wouldn't trust Windows RAID. And FlexRAID is a non-starter IMO due to limitations in FlexRAID View's design (that and it just doesn't seem like a "finished" product).

Though that only goes for "archived" type storage (ripped movies, etc). For recordings I agree that a couple local, high performance drives (I run a pair of Caviar Blacks) is the best bet.

Originally Posted by GKusnick

If you're serious about recording directly to NAS you will probably want to upgrade to gigabit LAN.

I'd upgrade regardless.

[sflamm]

The thing to remember with recording is that you need pretty good random I/O performance, so you'll probably want something rated or benched a lot higher than just those numbers would say.

Video recording is sequential writing... not random I/O. As is reading of video.

Cost of diskless internal NAS:

Internal enclosure: 4-in-3 $120
http://www.satagear.com/SATA-SRT43K_SATA_RACK_Case.html

Raid controller: $350
http://www.amazon.com/Controller-4-P.../dp/B000NX5BXK

Total cost: $470

A lot lower than external diskless raid boxes (I own a ReadyNAS btw)

Add disk to either solution as you desire. For instance, these Enterprise 1.5TB drives (I've used them) are $180 / disk -- and give 4.19TB of usable storage (A whole lot of recording)

http://www.newegg.com/Product/Produc...-787-_-Product

Enterprise disks are not required with this kind of controller. For instance these 1.5TB drives are $80 / disk work fine (I've used them)

http://www.newegg.com/Product/Produc...82E16822148337

[sflamm]

Though I recommend GigE to anyone as well...

Each 1080p stream (which is rare) is max 19.2 Mbits/s or 2.4MB/s.
Max throughput on a FE network is 12.5 MB/s with a realistic average of around 8 MB/s. (This is bi-directional)
That allows up to 3-4 full HD streams simultaneously. Again most broadcast HD is not 1080p and of course digital SD is also a lot lot lower.

The major constraint for success is the I/O of your server -- which is where the internal RAID bank plays such an important role.

With SageTV 6 the stream is recorded (written) to the disk and then played (read) from the disk to the client. So I/O rates can be double - though SageTV 7 may have improved on this, reading from memory... not sure. Anyone know?

[sflamm]

Forgot to mention - format the RAID bank using a GPT partition under windows.

[stanger89]

Originally Posted by sflamm

Video recording is sequential writing... not random I/O. As is reading of video.

In theory, but not reality. In reality after a "short" time of recording, playing, deleting, recording somemore, more deleting, etc, fragmentation means that the reads and writes are essentially random. This is one reason why Sage always used to recommend 64k clusters.

Cost of diskless internal NAS:

Internal enclosure: 4-in-3 $120
http://www.satagear.com/SATA-SRT43K_SATA_RACK_Case.html

Raid controller: $350
http://www.amazon.com/Controller-4-P.../dp/B000NX5BXK

Total cost: $470

A lot lower than external diskless raid boxes (I own a ReadyNAS btw)

That raid card is nearly twice as much as the Supermicro board I put in my unRAID box, and probably uses as much power. And I've got 5 disks in my unRAID box and can add another one "free" and another 8 on top of that for about $100 SATA card. So for the cost of that RAID card alone you can have a 14-drive capable unRAID.

FWIW, the cost of the latest gen of ReadyNAS's is why I don't have two.

[Fuzzy]

For starters, I don't use any sort of redundancy - I just don't see the need for media. I have nearly immediate off-site backup of anything important (photos, documents, etc). But frankly, I have no desire to 'protect' what are essentially already backups of my purchased media, or stuff I recorded off TV for single instance viewing. That said, I did use to use windows XP with software RAID 5, and it was indeed great. I actually prefer the way windows handles it's software RAID arrays over just about anything out there - the only problem being that they essentially removed the capability from win7. FlexRAID is 'different' in that it isn't real-time live redundancy. But, at least it works on the same machine. You are correct that it isn't finished - and I'm not really sure it will be. Though, as I already mentioned, I don't even see a need for it anyways.

[sflamm]

In theory, but not reality. In reality after a "short" time of recording, playing, deleting, recording somemore, more deleting, etc, fragmentation means that the reads and writes are essentially random. This is one reason why Sage always used to recommend 64k clusters.

In theory and in reality. Windows default is 4K block size. Moving to 64k provides a very moderate improvement. But not really anymore. The reality is with a reasonable size disk (which is easy to achieve now) and reasonably background defrag (i.e. Perfect Disk) -- most videos will be almost fully contiguous. If you have random writes of your video you have a serious issue

That raid card is nearly twice as much as the Supermicro board I put in my unRAID box, and probably uses as much power. And I've got 5 disks in my unRAID box and can add another one "free" and another 8 on top of that for about $100 SATA card. So for the cost of that RAID card alone you can have a 14-drive capable unRAID.

The feature set and performance of your unRAID box is very different from this real RAID solution (better than apple-to-apples with the feature set of ReadyNAS):

Key RAID Features
• Supports 4 direct-attached or up to 128 SAS or SATA disk drives using SAS expanders 
• RAID Levels 0, 1, 10, 5, 50*, JBOD
• Advanced Data Protection Suite
• RAID Levels 1E (Striped Mirror), 5EE (Hot Space), 6 and 60* (Dual Drive Failure Protection)
• Copyback Hot Spare
• Snapshot Backup (optional)
• Dynamic caching algorithm
• Online Capacity Expansion
• RAID Level Migration
• Optimized Disk Utilization
• Quick Initialization
• Native Command Queuing (NCQ)
• Hot spares – global, dedicated, and pooled
• Background initialization
• Automatic/manual rebuild of hot spares
• SAF-TE enclosure management support
• Configurable stripe size
• S.M.A.R.T. support
• Up to 512TB array sizes
• Multiple arrays per disk drive
• Bad stripe table
• Dynamic sector repair
• Staggered drive spin-up
• Bootable array support
• Hot-plug drive suppor

If you dont like that card you can go for a lower end Adaptec RAID card about $150.

As for power the PCIe RAID card is a power sipper @ 3.3W a lot lower than your solution - your power bill alone will cost you 10x than this whole solution over time.

a lot

[stanger89]

Originally Posted by Fuzzy

For starters, I don't use any sort of redundancy - I just don't see the need for media. I have nearly immediate off-site backup of anything important (photos, documents, etc).

OT, but what do you use for that?

But frankly, I have no desire to 'protect' what are essentially already backups of my purchased media, or stuff I recorded off TV for single instance viewing.

I don't have any of my recordings protected, but redundancy is a small price to pay for the insurance that I won't have to re-rip hundreds of movies, or with where I'm sort of going now, tons and tons of TV shows ripped from disc. And this is more of a problem as drive sizes increase. I don't want to lose 2TB of work, that's over 350 DVDs, 70BDs, or maybe 2000 (or more) TV episodes. That's tens if not hundreds of hours of work, and my time is valuable to me.

Originally Posted by sflamm

In theory and in reality. Windows default is 4K block size. Moving to 64k provides a very moderate improvement. But not really anymore. The reality is with a reasonable size disk (which is easy to achieve now) and reasonably background defrag (i.e. Perfect Disk) -- most videos will be almost fully contiguous. If you have random writes of your video you have a serious issue

That requires you to run extra background software. I am not assuming any extra software running. And of course even if you do run software like that, if you have multiple recordings and or multiple playbacks or a combination there of, you'll have effectively pseudo-random access anyway due to the nature of having to switch back and forth between the files you're reading/writing.

Point is in a multitasking DVR environment you can't count on sequential access.

The feature set and performance of your unRAID box is very different from this real RAID solution (better than apple-to-apples with the feature set of ReadyNAS):

I never said they were, I was comparing to Fuzzy's local storage. I tried to avoid another box for months if not years, and then I finally looked into unRAID a little deeper. I don't need RAID's sequential performance, and with an Supermicro X7SPA board, the unRAID is cheaper and about the same power as local RAID with a good RAID card.

[QUOTE]

Key RAID Features
• Supports 4 direct-attached or up to 128 SAS or SATA disk drives using SAS expanders 
• RAID Levels 0, 1, 10, 5, 50*, JBOD
• Advanced Data Protection Suite
• RAID Levels 1E (Striped Mirror), 5EE (Hot Space), 6 and 60* (Dual Drive Failure Protection)
• Copyback Hot Spare
• Snapshot Backup (optional)
• Dynamic caching algorithm
• Online Capacity Expansion
• RAID Level Migration
• Optimized Disk Utilization
• Quick Initialization
• Native Command Queuing (NCQ)
• Hot spares – global, dedicated, and pooled
• Background initialization
• Automatic/manual rebuild of hot spares
• SAF-TE enclosure management support
• Configurable stripe size
• S.M.A.R.T. support
• Up to 512TB array sizes
• Multiple arrays per disk drive
• Bad stripe table
• Dynamic sector repair
• Staggered drive spin-up
• Bootable array support
• Hot-plug drive suppor

And you need to have all the same sized drive (ideally same model), all drives must be spinning for the array to work, expansion are a relatively "big deal". And SAS expanders are rather expensive and not that easy to find.

If you dont like that card you can go for a lower end Adaptec RAID card about $150.

If I were buying a RAID card it would be a 3ware/LSI/Areca.

As for power the PCIe RAID card is a power sipper @ 3.3W a lot lower than your solution - your power bill alone will cost you 10x than this whole solution over time.

Not really, the X7SPA is only 20W by itself (including PSU inefficiencies), but on top of that only the needed drives are spun up (vs all drives in a RAID array).

RAID's great for some things, but a media server's requirements just don't line up terribly well with RAID's strengths.

[Fuzzy]

Originally Posted by stanger89

OT, but what do you use for that?

Mozy fits my needs perfectly for this.

[Nelbert]

tbh, home users paying that much for a raid card in a media nas is crazy.

If you want a raid card EOL/near EOL branded cards from ebay are much better value. HP/Intel/Sun etc... they're all rebranded LSI,Adaptec,3ware(LSI) anyway.

$165 got me a Sun StorageTek SAS RAID HBA delivered to the uk. 8 ports, 256meg cache, pcie x8, etc... current retail pack only comes with bbu now and costs ~$500

I've no desire to run a 2nd server, don't have a need for more than 8 drives and wouldn't trust mdadm or unraid with 16 drives added over a number of years anyway. With most hardware failure occuring at the beginning or end of a drives life there's an increased chance of multiple drive failure on older driver unless you add buy them 1 at a time or buy different brands and batches when buying multiple drives together.
Flexraid has potential, but without the releases with the new features that's all it has at the moment.
If all you want is a bunch of disks appearing in the same dir structure then unison/aufs is probably the easiest and best performing route.

As for sequential performance? How much sequential access do you expect to be doing when recording 2 or 3 things simultaneously, while having a movie streaming to one room and music or another recording streaming to another room? Even if the streaming reads are from sequential files, you're disks won't be seeing sequential access patterns...

[sflamm]

but redundancy is a small price to pay for the insurance that I won't have to re-rip hundreds of movies, or ...

correct. with modern disk drive sizes it makes no sense to run a platform without redundancy which is a small incremental cost - nothing compared to the loss of information in terms of its value and time to fix/replace

That requires you to run extra background software. I am not assuming any extra software running. And of course even if you do run software like that, if you have multiple recordings and or multiple playbacks or a combination there of, you'll have effectively pseudo-random access anyway due to the nature of having to switch back and forth between the files you're reading/writing.

Unlike an SSD, on a hard drive deletion the file system allows for the immediate re-use. If a drive is primarily dedicated to video then the open slot will typically be 2-6 GB and the 'splitting' of the video into different areas of a disk is low. It will not tend to fragment into small pieces.

If you dont want to pay for defrag, use the windows scheduler to do a defrag using the built-in windows defrag tool nightly at 2a (the defrag tool is actually an OEM of DiskKeeper and is good)

You ought to defrag no matter what - otherwise OS performance will vastly degrade (page file fragmentation etc...) and file system performance will degrade (small open sizes, etc). No matter what - you should be automatically defragging...

Lastly with a RAID 5, the issue is vastly reduced automatically because it is striped over multiple disks... both sequential and random access are vastly faster. Even random I/O on my bank is in excess of 100MB/s sustained (using a tool like HDTach to test)

Note you are wrong as to the reason why 64K block sizes were recommended anyway. The reason is no longer applicable since spindle speeds, platter densities and number of platters in a modern disk do not have the same constraints. Therefore there really is no advantage to a 64K block size.

I don't need RAID's sequential performance, and with an Supermicro X7SPA board, the unRAID is cheaper and about the same power as local RAID with a good RAID card.

Not true at all. The motherboard alone will consume much more than the RAID card... 3W for the RAID card versus minimum of 80W for any motherboard/CPU (btw, hard drives typically pull 8W each, but that would be the same in both scenarios).

and the total cost is not cheaper. Unraid pro server is $120 - the same cost as a low end Adaptec card. So $120 plus the cost of the motherboard (min $50), plus the cost of the CPU (min $50), plus the cost of the memory (min $50), plus the cost of the power supply (min $30), plus the cost of the chassis (min $50)... That's a minimum of $120 + $230 = $350. So it is about a wash on cost - but your solution requires a whole other box to maintain and costs more power consumption-wise to maintain and has far fewer features

And you need to have all the same sized drive (ideally same model), all drives must be spinning for the array to work, expansion are a relatively "big deal". And SAS expanders are rather expensive and not that easy to find.

Drives do not need to be the same size (the array will use the minimum common denominator of size).

It is recommended they have the same spindle rates, but not required.

Expansion is trivial. Hot swap out a drive and replace with a larger drive. Array will auto-rebuild.

No need for SAS. Works fine with all SATA. In reality you will not need more drives than 4. With off the shelf 2 and 3 TB drives now a single RAID bank will be 8 - 12TB (appears as a single drive under windows when formatted with a GPT file system). That is beyond any practical need.

[sflamm]

If I were buying a RAID card it would be a 3ware/LSI/Areca.

Not pitching any specific brand. The feature set / cost is what I am arguing - which really isnt subjective.

[stanger89]

Originally Posted by sflamm

Unlike an SSD, on a hard drive deletion the file system allows for the immediate re-use. If a drive is primarily dedicated to video then the open slot will typically be 2-6 GB and the 'splitting' of the video into different areas of a disk is low. It will not tend to fragment into small pieces.

Depends on how files are written, AFAIK Sage doesn't "reserve" 6GB or whatever for a recording, it allocates space incrementally, so yes, it does "split"

If you dont want to pay for defrag, use the windows scheduler to do a defrag using the built-in windows defrag tool nightly at 2a (the defrag tool is actually an OEM of DiskKeeper and is good)

But it's completely unnecessary for local discs, I've never defragged my recording drives in years of using Sage.

My point though was that if you're recording to a NAS, you want one with good performance. Lots of NAS's have horrible performance, nothing close to a local disk. So if you're recording to a NAS you want to make sure to get a relatively high performance one. Oh and defrag is moot on a NAS, you can't do it unless you're doing iSCSI or something.

You ought to defrag no matter what - otherwise OS performance will vastly degrade (page file fragmentation etc...) and file system performance will degrade (small open sizes, etc). No matter what - you should be automatically defragging...

There is absolutely no need to do it on a recording or media drive.

Lastly with a RAID 5, the issue is vastly reduced automatically because it is striped over multiple disks... both sequential and random access are vastly faster. Even random I/O on my bank is in excess of 100MB/s sustained (using a tool like HDTach to test)

Try Crystal Diskmark on it with 4k size. Most SSDs can't hit 100MB/sec @4k.

Note you are wrong as to the reason why 64K block sizes were recommended anyway. The reason is no longer applicable since spindle speeds, platter densities and number of platters in a modern disk do not have the same constraints. Therefore there really is no advantage to a 64K block size.

Sorry but this is simply incorrect, 64k clusters were recommended for performance reasons, due to increasing disk performance. SageTV made software changes in V7 such that this is no longer needed.

Not true at all. The motherboard alone will consume much more than the RAID card... 3W for the RAID card versus minimum of 80W for any motherboard/CPU (btw, hard drives typically pull 8W each, but that would be the same in both scenarios).

The X7SPA installed in a case with PSU pulls ~20W total.

So it is about a wash on cost - but your solution requires a whole other box to maintain and costs more power consumption-wise to maintain and has far fewer features

I don't know what more there is to maintain, I don't "maintain" my unRAID box any more than the RAID-5 array in my old server. And as far as power goes, I went from 2 boxes: (Sage server with 8 drive RAID-5 array on a 3ware card + 2 caviar blacks + a couple other drives) and a ReadyNAS X7, to 3 boxes: unRAID (X7SPA w/ 5 drives), same ReadyNAS, new Sage Server, and dropped 50-100W off my power usage. The whole "farm" is under 300W now (including routers, switches and Sat boxes), where is was closer to 400W before.

Drives do not need to be the same size (the array will use the minimum common denominator of size).

And you waste the space on the larger drives, or you have to create a second volume/array on the spare space on the larger drives (if your card allows it, and I think only the more expensive ones tend to do that), and if you upgrade your drives, you have to manually expand or create new volumes. And then expand the partitions.

unRAID you just plug in new drives or replace an old one with a larger one (though you do have to have the parity drive be the largest).

It is recommended they have the same spindle rates, but not required.

And you kill half the point of a RAID-5 array if you try that.

Expansion is trivial. Hot swap out a drive and replace with a larger drive. Array will auto-rebuild.

So you're telling me if I have a 4x 1.5TB array (4.5TB total space), and I replace 2 of the drives with 2.0TB drives the array would automatically expand to 5TB? I don't think so, not on any RAID-5 system I've seen. Any individual unit on the card is limited by the smallest drive in the unit.

To expand with larger drives, you'd end up having to make a second RAID 1 volume on the extra 500GB of each drive, and then you'd have two units on the card/drives. When you add your third 2.0TB drive, you'd have to RLM the RAID-1 to RAID-5, but you'd still have two units, and I've not heard of any cards allowing you to merge units.

No need for SAS. Works fine with all SATA. In reality you will not need more drives than 4.

You mentioned SAS expanders as a benefit of a hardware RAID card, that's why I mentioned them. Have you tried to find one? Easier said than done. As far as number of drives, I've got 12TB of redundant storage, how do I do that today with 4 drives? Best you could do is 9TB and that's by buying external drives and ripping the HDDs out of them.

With off the shelf 3 and 4 TB drives now a single RAID bank will be 12 - 16TB (appears as a single drive under windows when formatted with a GPT file system). That is beyond any practical need.

But you can't buy 4TB drives aren't out, and you can't buy 3TB drives yet standalone that I've seen. And with Blu-ray's being 25-50GB a piece, space goes quick.

Look, I'm not unfamiliar with RAID cards, I've got one, I've looked at all this, like I said I spend the better part of a year on and off trying to figure out how to solve my running out of storage problems. I tried very hard to avoid unRAID, but in the end, the things I was worried about (more power, an extra box, more managment, higher cost) all turned out to be non-issues.

Originally Posted by sflamm

Not pitching any specific brand. The feature set / cost is what I am arguing - which really isnt subjective.

My point is the cards I've looked at for the features I required of them were Areca/LSI/3ware cards and they were not cheap (like the $150 you mention), think $300-400 for a 4 port. And as I've illustrated, 4 drives isn't enough, I'd need at least 8, in fact I was looking at SAS cards (eg 3ware 9690-4I), which comes out to be about $350 + $225 for the SAS expander, so that's $575 just in RAID hardware, or $500 or so for an 8-port card.

[Nelbert]

Originally Posted by sflamm

Not pitching any specific brand. The feature set / cost is what I am arguing - which really isnt subjective.

Sorry, but feature set/cost is always subjective and can't be used as a reliable metric for choice unless "mine is bigger than yours" is the most important factor for the person buying.

TV A & B cost the same. TV A has 10 features, TV B has 50 features. I only require 4 features that both items have. For me, TV A has 6 irrelevant features and TV B has 46 irrelevant features.

Should I choose TV B because it has more features I won't use?
Or TV A because it has less features I won't use?

Or is the feature set/cost actually irrelavent because they both exceed my needs and things like design, build, engineering quality, ease of use, after sale support are more relevant.
Unless I'm cool and hip at which point the power of marketing and brand name will dictate all and I'll buy TV C with no features I want at twice the price.

[Spectrum]

Originally Posted by stanger89

But you can't buy 4TB drives aren't out, and you can't buy 3TB drives yet standalone that I've seen. And with Blu-ray's being 25-50GB a piece, space goes quick.

WD has put the 3TB drives on the market. They're a bit spendy though and not all SATA controllers will support them; it's new tech and it will be awhile before things settle down.