Need help building WHOLE house Media Server for 5 STX-HD100 Extenders

[Raystream]

Currently I have a Athlon 2600+, 512MB, 160GB HDD with a PVR 150 that I am planning on replacing it right now... but I would like to get the Tuners first before anything. (Within the next two weeks!) Below is my current wants.

OS: Windows XP
Motherboard: Intel Quad Core suported mobo
Additional: 2 eSata Ports, mutiple USB 2.0 ports, Dual Gigabit Network Ports
CPU: Intel Quad Core CPU (Not a clue how powerful of one to get)
Memory: 4-8GB of DDR3
Video Card: Nvidia 7600GT (Too Powerful?)
Hard Drive: 2 TeraBytes (16MB cache, 7200rpm Sata's) Expandable up to 6TB

Now what I really need to know about right now...
Tuners:
5 OTA ATSC HD Tuners
5 QAM Tuners

Currently I have been looking at the Hauppauge 1600 and 1800 cards. Should I be looking at PCI Express Tuner cards or USB external Tuners or the HDHomeRun Tuner?

I would like to get the best Dual Tuner cards... and I don't have need for any NTSC analog tuners.

[Diginerd]

Here's a bunch of observations & questions...

I have to ask why that mix of tuners?

Personally if you did need to grab such a large amount of unencrypted TV I'd grab 2-3 HD Home runs, each supporting 2 streams of either OTA or Cable digital. The nice thing about them is they are network devices and hence require no slots in your server. :-)

If you have more than 3GB of RAM in your server you'll need XP64 or a Vista 64 edition, which can potentially cause you all kinds of driver headaches, not to mention software incompatability issues. 3Gb is more than enough.

If you are running your server "Headless" and are not doing on the fly transcoding (Which if you have HDX-1000s you prolly won't) then onboard graphics will probabbly get you by as you'll never display video on that box.

If you're recording lots of streams of HD I'd strongly advise going with a RAID array. RAID 0 if you are after pure speed, but don't care about doubling your chance of a hardware failure, or RAID5 is you want some resiliency with the benefit of multiple spindles handling your throughput.

Your network had better be GigaBit giving the amount of traffic you're pushing around.

Seriously consider putting in multiple storage boxes to split up the load, as well as to Comskip. Relieving the primary server of such duties will definately help your playback of multiple streams as well as recording at the same time. With few drives on one machine you're going to put yourself in the hurt locker very quickly with drive contention & thread locking.

All this has prompted me to sit down and document how i built my large rig. I think there is some benefit to others in seeing how after many years of searching I've finally hit a happy place with a large setup.

[CollinR]

Yes odd mix of tuners selected with no baseband stuff as well is strange.

On board video is dandy, you only need to get through post and most won't with no display adapter at all.

Again a RAID array but I would look at a nested version like RAID10 but you would need more disks. Right now 4 500GBs is cheaper then 2 1TBs and you can get 500GB SV35 drives where you cannot 1TB.

Offload commskip.


You should consider R5000HD Vip211 boxes and Dishnet, more $ but x10 the content.

[BFisher]

I have 5 Extenders (currently 4 MVPs and 1 HD) with my headless server... to be honest I think you will have less server needs with 5 HD Extenders than MVPs (no more server transcoding).

I have a Quad Core, 2 GB Ram, Gigabyte (ultra durable/cool) Mobo, 1.7 TB harddisk space, and 6 tuners (mix of OTA HD and DirecTV SD).

I didn't bother with RAID - disk speed has never been an issue.

I've had all 5 working, including HD to SD transcoding, recording 3-4 channels, commercial skip processing, etc and never had an issue with processor capability or bandwidth. I think 5 HD extenders will only make it easier on your server... not harder

[mikesm]

Software RAID under XP is asking for trouble. Just set it up as inidividual volumes and let Sage do the storage management.

thx
mike

[CollinR]

Originally Posted by BFisher

I think 5 HD extenders will only make it easier on your server... not harder

No you are correct, disk access and network throughput is basically the only concern.

However transcoding for storage will have some tax, it comes down to how much and how often. If you use the auto compress plugin it will keep your CPU busy with possibly 10 videos.


____________


You do realize you will almost have a tuner for like every other or third channel in your lineup?

With OTA (usually pointless with a QAM tuner) and QAM I would be surprised if you have enough unique content feeds. If you put in a PVR500 you would at least get 2 more feeds of the first ~70 analog SD channels.

[kevine]

Originally Posted by CollinR

Offload commskip.

Do many people do this? I picture this as another PC that reads the MPEG2 off the server, then to create the comskip edl file back on the server. Basically this reserves the server CPU for things that cannot but offloaded like transcoding, autocompress, etc. Is this what you mean? I never thought of doing this.

[NEOSG]

Originally Posted by kevine

Do many people do this? I picture this as another PC that reads the MPEG2 off the server, then to create the comskip edl file back on the server. Basically this reserves the server CPU for things that cannot but offloaded like transcoding, autocompress, etc. Is this what you mean? I never thought of doing this.

I started offloading comskip when i went from sage 5.x to 6.x. For some reason when i made that move, the server cpu could just not handle doing it all if there were a lot of recordings going on at the same time as files being processed. Since then, all is well. AMD64 3200+ is the processor of the sage server. Don't remember what the comskip processing box is off the top of my head

[BFisher]

I've watched, when comskip is working (actually, I use Studio Analzer), my CPU % barely even changes. It's so little that I've never considered off-loading... even when recording 3 or 4 things at once.

[mikesm]

Originally Posted by kevine

Do many people do this? I picture this as another PC that reads the MPEG2 off the server, then to create the comskip edl file back on the server. Basically this reserves the server CPU for things that cannot but offloaded like transcoding, autocompress, etc. Is this what you mean? I never thought of doing this.

I do this with showanalyzer. I have a server that is lightly loaded but runs exchange etc... and I have it run showanalyzer processing to keep the Sage server CPU available for transcoding.

Even better to run it on the NAS server if you can get away with that.

PS If you CPU doesn't get busy running the commercial detection, maybe you have an I/O bottleneck making the analyzer software wait...

thx
mike

[lobosrul]

If I won the lottery I'd go with the following:

Intel QX9650
4 GB RAM DDR3
32 GB of solid state storage as my boot drive
6 1 TB Drives in 3 seperate mirrors
Tuners: probably 3 VBOX dual ATSC pci-e cards
3 or 4 r5000 devices for Dish Network (whatever the maximum # supported is)

Plus one of these: http://www.newegg.com/Product/Produc...82E16822102004 in raid-5 mode, for TV archival (after transcoding), DVD storage etc. Connected thru USB so as to not clog ethernet connection.

Gigabit would be a must. Plus some sort of wireless transmitter in the attic.

In bedrooms I would have the extender, but I think on the big TV I would still keep an actual client just in case theres something out there I want to watch but the extender doesn't support.

Mobo: not sure exactly, probably the newest NVIDIA chipset. I know I'll never buy a Gigabyte brand again. MSI or ASUS for me.

[Crashless]

If you're looking at 10 HD recordings and 5 playback at once, you're going to HAVE to get a good RAID card and Gigabit ethernet.

Quick math:
20mbps per HD stream x 15 = 300mbps = 37.5MBps

There aren't many single hard drives that can sustain that kind of random throughput - it is very different than sustaining 40MBps in one continuous file.

I'd estimate 3-4 drives in RAID 5 would be a good start. You could probably get away with 3 in RAID 5 or 0, but I think even 2 drives in RAID 0 would cause issues - especially if you plan on running any commercial detection.

Good luck!

[mikesm]

Originally Posted by Crashless

If you're looking at 10 HD recordings and 5 playback at once, you're going to HAVE to get a good RAID card and Gigabit ethernet.

Quick math:
20mbps per HD stream x 15 = 300mbps = 37.5MBps

There aren't many single hard drives that can sustain that kind of random throughput - it is very different than sustaining 40MBps in one continuous file.

I'd estimate 3-4 drives in RAID 5 would be a good start. You could probably get away with 3 in RAID 5 or 0, but I think even 2 drives in RAID 0 would cause issues - especially if you plan on running any commercial detection.

Good luck!

My Linux NAS can do that with software raid. But yes, you will need to tune it for good performance, and yes, 1 disk may be hard pressed to handle it.

[pez]

Originally Posted by BFisher

I've watched, when comskip is working (actually, I use Studio Analzer), my CPU % barely even changes. It's so little that I've never considered off-loading... even when recording 3 or 4 things at once.

Is Studio Analyzer setup to run only run at the frame rate of the recorded video? So that an hour show takes an hour to detect the commercials. ShowAnalyzer runs as fast as it can. Even on my slow (really slow) SW RAID SA can peg a 2GHz P4.

[briands]

Originally Posted by Crashless

If you're looking at 10 HD recordings and 5 playback at once, you're going to HAVE to get a good RAID card and Gigabit ethernet.

Quick math:
20mbps per HD stream x 15 = 300mbps = 37.5MBps

There aren't many single hard drives that can sustain that kind of random throughput - it is very different than sustaining 40MBps in one continuous file.

I'd estimate 3-4 drives in RAID 5 would be a good start. You could probably get away with 3 in RAID 5 or 0, but I think even 2 drives in RAID 0 would cause issues - especially if you plan on running any commercial detection.

Good luck!

With Sage's drive balancing, you will not likely be writing all of the shows to the same disk though, right? Unless you add a new drive...

[GKusnick]

Originally Posted by briands

With Sage's drive balancing, you will not likely be writing all of the shows to the same disk though, right? Unless you add a new drive...

It's not the writing that matters so much as the reading. Recording several shows to the same disk simultaneously will result in fragmentation, but not that much seeking, since they can all be written to roughly the same area of the disk. Simultaneous playback is a different story; you have no choice but to read from wherever the files happen to be, so you get a lot more seeking, which is what kills throughput.

[Raystream]

Originally Posted by Diginerd

Here's a bunch of observations & questions...

I have to ask why that mix of tuners?

Personally if you did need to grab such a large amount of unencrypted TV I'd grab 2-3 HD Home runs, each supporting 2 streams of either OTA or Cable digital. The nice thing about them is they are network devices and hence require no slots in your server. :-)

If you have more than 3GB of RAM in your server you'll need XP64 or a Vista 64 edition, which can potentially cause you all kinds of driver headaches, not to mention software incompatability issues. 3Gb is more than enough.

If you are running your server "Headless" and are not doing on the fly transcoding (Which if you have HDX-1000s you prolly won't) then onboard graphics will probabbly get you by as you'll never display video on that box.

If you're recording lots of streams of HD I'd strongly advise going with a RAID array. RAID 0 if you are after pure speed, but don't care about doubling your chance of a hardware failure, or RAID5 is you want some resiliency with the benefit of multiple spindles handling your throughput.

Your network had better be GigaBit giving the amount of traffic you're pushing around.

Seriously consider putting in multiple storage boxes to split up the load, as well as to Comskip. Relieving the primary server of such duties will definately help your playback of multiple streams as well as recording at the same time. With few drives on one machine you're going to put yourself in the hurt locker very quickly with drive contention & thread locking.

All this has prompted me to sit down and document how i built my large rig. I think there is some benefit to others in seeing how after many years of searching I've finally hit a happy place with a large setup.

Is there a limit in SageTV as to how many HD Homeruns I can have? Also, on one HDHomerun do I have to hook up both connections to either OTA or QAM or can I mix? Currently I have Cat6 Shielded 10GB certified cable run and a 24 port 1 Gigabit switch so there is plenty of room to expand.

Currently I have Comcast basic cable, Skyangel via Dish Network and a UHF HD antenna. I'm really not interested in Dish networks regular packages. So that leaves Comcast and the antenna. And I'm really not interested recording analog channels from comcast so hence the 5 QAM Tuners that I want to get.

If I recall for XP x86 32bit there is a patch from Microsoft to support larger then 3Gb... more so a cludge patch. I'm just worried about what the future might hold so I would rather be safe then sorry and get at least 4GB of ram.

As far as hard drives go... I am looking at getting 5 of the following, and I think I will put them in Raid 5.

SAMSUNG Spinpoint F1 HD753LJ 750GB 7200 RPM 32MB Cache SATA 3.0Gb/s Hard Drive
http://www.newegg.com/Product/Produc...82E16822152100

As long as I get a motherboard with 2 eSata ports I am thinking of getting the following eSata 5 port hub. Has anyone used one of these?
http://www.cooldrives.com/sahub5muussi.html

So what should I do about internal tuner cards? Are the Hauppauge 1600 and 1800 cards any good? What about ATI TV Wonder™ Digital Cable Tuner or the ATI TV Wonder™ 650 Combo USB for PC ? -> http://ati.amd.com/products/tvwonder650/usb/index.html.

As far as what someone mentioned about having an offloading storage are you referring to another Sagetv server or something like the following? -> http://www.newegg.com/Product/Produc...ORCO%2bDS-1220

[bcjenkins]

Originally Posted by kevine

Do many people do this? I picture this as another PC that reads the MPEG2 off the server, then to create the comskip edl file back on the server. Basically this reserves the server CPU for things that cannot but offloaded like transcoding, autocompress, etc. Is this what you mean? I never thought of doing this.

I use my theater room PC to do comskip as it sits idle most times. connected via gb net of course.

B

[BFisher]

Originally Posted by pez

Is Studio Analyzer setup to run only run at the frame rate of the recorded video? So that an hour show takes an hour to detect the commercials. ShowAnalyzer runs as fast as it can. Even on my slow (really slow) SW RAID SA can peg a 2GHz P4.

I put no limitations on Show Analyzer. However, I have it working while it's recording so it's only going as fast as the show is being recorded... but the nice part is commercial skipping is available on shows that are being watched in real-time (well, time-shifted some obviously (can't skip commercials that aren't there yet )

[briands]

Originally Posted by BFisher

I put no limitations on Show Analyzer. However, I have it working while it's recording so it's only going as fast as the show is being recorded... but the nice part is commercial skipping is available on shows that are being watched in real-time (well, time-shifted some obviously (can't skip commercials that aren't there yet )

I thought I had this working this way as well, but I've noticed that the updates to the edl file are not sent to the player unless you stop and restart the player. Is there a setting somewhere?