NAS and HD Recordings

[RayN]

I just finished building an NAS box using NASLite-2 USB as the OS. It contains 4 250 gB WD PATA drives. I put it in the basement about 60 ft away connected by Cat 5 cable.

SD recordings record and playback just fine. HD is another story. I get severe pixelization, broken audio and the timeline jumps in 2 to 3 second steps at times during playback.

The problem seems to occur during recording. If I copy one of the recorded files from the NAS to my local box and play it back, the same playback issues remain.

I moved the NAS next to my main HTPC, actually a combination HTPC and general purpose box, and connected it to my router with a 6 ft. cable. The situation improved somewhat, since I am able to record and playback some HD programs that are watchable, but not others. I've only had it up for a day so I don't have much experience with it.

My questions:

1. Will upgrading my network to Gigabit be helpful? How would I do that? I am trying to keep costs down.

2. NAS Lite-2 formats the disk with 4K clusters with no options available. The two SATA drives that I have in my main box are formatted with 64K clusters and they work fine with HD recordings.

3. If 64K are a must, could I run a Windows OS on the NAS instead of NASLite-2, reformat the drives to 64K and keep the network cable length short without upgrading to Gigabit?

[lovingHDTV]

I've not found anyone who uses any NAS that can successfully stream HD recordings cleanly. I've asked several times on this forum, but noone that has a NAS ever uses it for HD. Typically they use it for DVD archival, or SD recordings only.

The NAS I use is UnRaid by Lime-Technology. They just released a new version with performance improvements and it has been reported by one user that he can successfully stream two HD shows, while recording a third all simultanesouly. I've personally not tried this, but will when I get around to it.

Currently my solution is to use the NAS for DVD and SD recordings, while I have a local drive in my Sage server for HD recordings only. I setup Sage such that the local drive is only used by my HDTV tuner.

Here is a link to the UnRaid performance reports:
http://support.lime-technology.com/f...hp?topic=220.0

His report is towards the end of the first page.

Maybe this helps?

[RayN]

Yes, Dave, it does help.

My next question is how do I tell Sage to record HD to my local drives only? I've search the forums and haven't seen the method described.

Can you fill me in?

Ray

[Kirby]

Look in your mmc/encoder lines for a specific tuner (in your sage.properties file), there is a line like "mmc/encoders/12345/forced_video_prefix=".

Set it to whatever location you want. You can also force each tuner to a specific drive or UNC path.. Very handy feature.

[jquinlan]

Originally Posted by RayN

My questions:

1. Will upgrading my network to Gigabit be helpful? How would I do that? I am trying to keep costs down.

2. NAS Lite-2 formats the disk with 4K clusters with no options available. The two SATA drives that I have in my main box are formatted with 64K clusters and they work fine with HD recordings.

3. If 64K are a must, could I run a Windows OS on the NAS instead of NASLite-2, reformat the drives to 64K and keep the network cable length short without upgrading to Gigabit?

I would think Gigabit would help but I don’t know if it will fix all problems. 64K clusters have always been a must in discussions but I understand your dilemma.
I currently use my NAS, Win 2000 Server with 64K clusters 100 Megabit network, for DVD and SD's pulled from SAGE for archive. I don’t have SAGE record directly to the NAS, only to its local drive; I use the NAS for recording that I wish to keep since the NAS is using RAID 1 and this time, and the SAGE server does not RAID. Since it is working I was going to move to RAID 5 to increase storage, but I have also been waiting for a good HD extender to then test HD programs while making the upgrades for the best results.

Sorry I can’t help with a definite answer because I also am not doing HD records yet.

On question 3 what if you install Gigabit into the server and the NAS and use a crossover cable. This way the Gigabit will only be use for video traffic and your 100 Megabit connections with your router can be used for clients. Dual networks with a minimal cost of just the cards.

[jlabrue]

I have a Windows server configured with a Raid Array in my Study. My HTPC box is in my living room. They are both connected by fairly long runs (25-50ft) and plugged into the same switch. I AM using Gb for both devices and I can record 2 HD streams and 2-3 SD streams simultaneously across the network to my NAS box with rarely (not never ) a problem. I am using 64k clusters. I have also recorded 2 HD and 1-2 SD programs while watching a recording on an MVP in a bedroom at the same time.

So, HD can be recorded over a network to a NAS box but Gb network is essential to success. You can get a Gigabit switch for next to nothing to accomplish this task. Disk I/O is also important for multiple streams. I am using a HW based SATA Raid controller with a RAID3 array with 5 drives.

[Motofreak75]

Originally Posted by lovingHDTV

I've not found anyone who uses any NAS that can successfully stream HD recordings cleanly. I've asked several times on this forum, but noone that has a NAS ever uses it for HD. Typically they use it for DVD archival, or SD recordings only.

Hello there. I have my recording directory setup to use my two pata drives using NAS2 cdrom on 100mbit card while hooked up to a gigbit network. I dont have a gigbit card that is support by NAS2 yet. BUt my NAS server specs are

P4 1.6 with 512 pc133 ram, 100Mbit intel card & 2x300 gig pata drives & a few sata 500gig drives with about 150 hours of HD recording and has no problem playing back and HD recordings or watching live HDtv.

RayN
What are your hardware spec of your NAS server have you looked at the system info page on NAS to see the server load. HD is what 15Mbits/sec ? if you think your 100mbits isnt fast enough, you must have an error somewhere. I can stream 1080i HD through wifi (within 20 feet LOS of the access point) and the network load is 22% on a 54g setup. there must be something wrong, maybe your NIC is connectting at 10mbits ? which will cause it to fall on its face. Cables perhaps but I would start looking at software and hardware setups on both ends first before upgrading.

As I write this, I am recording two HD shows to my NAS and watching another from my NAS with no studdering etc..

[RayN]

Gentleman All,

Thanks for your replies. From the replies, it seems this HD thing can be done to a remote server with attention to the proper details.

Motofreak75,

My NAS server is a 3.2 P4 Celeron with 512mB PC2700 RAM, onboard 100mB network and 4X 250 PATA drives running NASLite-2 USB.

The network sttus info screen tells me that the NAS NIC is running 100 Mbps full duplex. I'm not sure where to look to find what the network transfer rate is.

I am going to start by designating my 2 X 250 SATA local drives for HD recordings. However, to do that, I need to find out how to tell Sage to move my current library of SD recordings to the NAS and still keep track of them. Hope that it's in the manual somewhere.

Kirby,

I've modified the sage properties file per your example. I got it to work for one drive. What is the syntax for designating either of 2 drives?

[Lucas]

Originally Posted by lovingHDTV

I've not found anyone who uses any NAS that can successfully stream HD recordings cleanly. I've asked several times on this forum, but noone that has a NAS ever uses it for HD. Typically they use it for DVD archival, or SD recordings only.

I set up a NAS server with NASLite V2 4 months ago and I also had problems with HD streaming initially.

I currently have no problems with HD on a 100Mbit network.

I found that the 64K cluster recommendation is only applicable when one records more than one video concurrently to the same disk. This because when two files are recorded concurrently the segments are recorded in an fragmented manner. With 64K clusters, the segments are large enough to enable the disk seeks to catch up.
Even then when the disk is near full and after a few months of recording, deleting etc, the fragmentation can still cause problems as I found out. I had to move all files to another disk and the copy them back, essentially removing the fragmentation.

From the above, it is easy to see that when recording to NAS with 4K clusters there will be problems.

There are two solutions:
1. Record all content locally and later move it file by file to the NAS.
2. Make sure each capture device points to a different NAS disk.

Either way the results are contiguous files on the NAS which should stream well.

I do 1. above. I record to a local 500GB disk and at a later time move the files one by one to the NASLite drive. I currently have over 1 TB of .ts files on the NASlite box that play well.

On my 100Mbit network I can stream a max of two concurrent HD streams from the NASlite without any issues. The third causes some hicups due to the bandwith limitation.

[RayN]

Originally Posted by Lucas

I set up a NAS server with NASLite V2 4 months ago and I also had problems with HD streaming initially.

I currently have no problems with HD on a 100Mbit network.

I found that the 64K cluster recommendation is only applicable when one records more than one video concurrently to the same disk. This because when two files are recorded concurrently the segments are recorded in an fragmented manner. With 64K clusters, the segments are large enough to enable the disk seeks to catch up.
Even then when the disk is near full and after a few months of recording, deleting etc, the fragmentation can still cause problems as I found out. I had to move all files to another disk and the copy them back, essentially removing the fragmentation.

From the above, it is easy to see that when recording to NAS with 4K clusters there will be problems.

There are two solutions:
1. Record all content locally and later move it file by file to the NAS.
2. Make sure each capture device points to a different NAS disk.

Either way the results are contiguous files on the NAS which should stream well.

I do 1. above. I record to a local 500GB disk and at a later time move the files one by one to the NASLite drive. I currently have over 1 TB of .ts files on the NASlite box that play well.

On my 100Mbit network I can stream a max of two concurrent HD streams from the NASlite without any issues. The third causes some hicups due to the bandwith limitation.

Lucas,

My initial HD recordings were to virgin disks in my NAS and were recorded one at a time, at 8 and 9 PM. Neither were playable from the NAS. I later moved to my local drive and they still did not play. Fragmentation was not the issue. Maybe my DVICO card had a hiccup that night.

I have now designated one of my local drives as the destination for my DVICO card's HD recordings and those drives have always worked great. Do you know how to tell Sage to use either of two drives for a given capture device? Is this possible?

I moved one HD file recorded locally last night to an NAS drive and it played back perfectly. I checked the network utilization in Task Manager during playback and it was in the range of 7% to 12%. It would seem that I could easily playback and record simultaneously without running out of network resources. I am at a loss to explain my initial experience.

I'm going record HD locally for now and move files to NAS as I need space.

[lovingHDTV]

RayN,
This has been my exact experience, except I have a gige connection between my NAS and SageServer.

It appeared to me as if the Sage write buffer was overflowing and creating bad recordings. I've not had issues with local drives.

[RayN]

Originally Posted by jds23

The cheapest thing you can do for starters is re-crimp your RJ45 cable ends. You can run a CAT5 cable around 335 ft. without needing repeaters, and 60+ ft. should work nicely if everything is perfect. One teeny bit of crosstalk could cause the problem you are explaining. It may/may not fix your problem, but it is cheap and easy to do, and you would not believe the amount of times I have seen this to be the problem!

Do I need a specialized tool to recrimp or can I just give a gentle squeeze with a pair of parallel jaw pliers?

[Lucas]

RayN, you will need an RJ45 crimping tool.

Also to enforce recording to a particular directory use:

mmc/encoders/12346/forced_video_storage_path_prefix=E

where 12346 is the numeric code in your Sage.Properties for your Encoder and E is the first character of an already defined(in Setup - Detailed) SageTV Recordings Directory.

So, if you already have a recording dir defined as E:\Recordings
all recordings from the encoder 12346 will be placed there.

[RayN]

Originally Posted by Lucas

RayN, you will need an RJ45 crimping tool.

Also to enforce recording to a particular directory use:

mmc/encoders/12346/forced_video_storage_path_prefix=E

where 12346 is the numeric code in your Sage.Properties for your Encoder and E is the first character of an already defined(in Setup - Detailed) SageTV Recordings Directory.

So, if you already have a recording dir defined as E:\Recordings
all recordings from the encoder 12346 will be placed there.

If Radio Shack has a reasonably priced one, I'll pick it up. Otherwise I'll see if I can borrow one.

I set up each of my sources to their own drives, HD to the one local drive and SD directly to the NAS.

[Jesse]

RayN, you will need an RJ45 crimping tool.

They can be had for cheap at Home Depot and Lowes.

Jesse

[kltye]

I actually managed to solve this problem for my setup. I have gigE cards on both the Sage server and my NAS. What I did was to raise the Receive Window (RWIN) on both ends to something like 262144, and the MTU to be 8192. I'm using Belkin and D-stink cards, and a cheapo gigabit Netgear switch. No more stuttering for me when recording to the NAS.

To change those settings in Windows, I used Dr. TCP (google for it; it's from dslreports). Not sure how to do it in Linux though :P

[stevech]

Have I correcty deduced that these cheap NAS boxes running an embedded Linux on a slow processor, and with small clusters, is just too underpowered?

I.e., maybe can cope with one std def TV stream on 100BT LAN, but no more. Some of these low cost NASes run inefficient file systems, I've read.

[lotusvball]

Originally Posted by stevech

Some of these low cost NASes run inefficient file systems, I've read.

From what I've read ext3 file system is actually more effecient then NTFS that windows runs. I know that NAS-lite+ uses ext3.

[Lucas]

Originally Posted by stevech

Have I correcty deduced that these cheap NAS boxes running an embedded Linux on a slow processor, and with small clusters, is just too underpowered?

I.e., maybe can cope with one std def TV stream on 100BT LAN, but no more. Some of these low cost NASes run inefficient file systems, I've read.

It depends on which low cost NAS boxes you are referring to.
If you mean these packaged products then it may be the case since I have also read that their network throuput is very low.

If you are talking about NASlite on a standard (even old PC with 1GHz CPU) then this is not the case.