New SageTV server build ideas?

[najames]

I'll try this again, hopefully it goes w/o timing out.

I have a WHS Sagetv server built with a mATX board, onboard graphics, Intel Q6600 CPU, 4GB RAM, 2 WD black 1TB and 2 WB Geeen 750GB drives. The OS drive is basically empty, but the others are now over 80% full. The server uses 90W at idle, measured last night with the Kill-a-watt meter. I minimumly need some 2TB drives to replace or add in.

I also need a backup system for other PCs. At first I was thinking of UNRaid or something similar, building with an Atom based PC. Then I thought, why not just combine them on the same box, perhaps using virtual machines? Linux vs WHS Vail? I am familiar with Linux too.

How would you go about building another new SageTV server, central storage repository, and backup system? I'd really like to do something that consumes less power if possible.

I have several other PCs already, including a Win7 Core i7 920 with 12GB that is not in use. Could it be used to access the Sage recordings for some form of compression/comskip if the server CPU was not up to the task?

I will likely have to build and test it, somehow move recordings, then replace the current server so we don't have any lapses to maintain already low WAF due to the now troublesome HDPVR. I can NOT miss any NFL games either.

[gplasky]

Well as you know WHS IS your system to baclup all other PCs on your network. So maybe look at a low power mb with a bigger case to hold more drives. Consider hot swap trays if it doesn't increase the cost too much. Use your more powerful, other PC on your network to do your comskipping. I think transcoding has to happen on the server unless you use some other external tool to compress/convert your recordings. And then you could run that on another PC. Just map a drive over to the Sage server. That's what I do.

Gerry

[stanger89]

I think I'm probably starting to look a bit like Duke Nukem forever, continually throwing out teasers for a project which never comes to pass, that said, I've been thinking of something very similar. I'm running out of storage, and my current system is quite inefficient, including 8x250GB drives in one array.

Anyway, my research always lead me back to unRAID, as being my ideal storage system, except for the fact that it needs it's own machine. Further reading and "research" led me to the possibility of running both Sage (in Windows) and unRAID on the same machine in virtual machines under the (free) Vmware ESXi.

The project didn't really go too far until the release of ESXi 4.1 just a short while ago. Until then the possibility of doing this existed with VMDirectPath (PCI passthrough), allowing the passthrough of tuners. But the hardware requirements made it very hard to test. But ESXi 4.1 added USB passthrough which seems to have much broader hardware support.

Long story short, I have built a proof of concept machine, my old HTPC hardware, Athlon BE-2400, Gigabyte ATI780G motherboard, running ESXi 4.1 with unRAID VM (using 3 passed-through disks), and a Windows XP SP3 VM for SageTV. I managed to get my HDHomeRun, HD PVR, USB-UIRT and R5000 all installed and functional in that configuration.

I even tested (thought probably wouldn't do in practice) recording to the unRAID VM and it appeared to work fine.

If I were buying hardware for this, I would strongly consider the Core i5. They're pretty powerful CPUs, yet Tom's HW managed to build an i5 system that pulled only 25W. On top of that it supports VT-d (VMDirectPath) with the right motherboard which would open up PCI passthrough to allow using PCI tuners in addition to just USB/network ones.

Originally Posted by najames

I have several other PCs already, including a Win7 Core i7 920 with 12GB that is not in use. Could it be used to access the Sage recordings for some form of compression/comskip if the server CPU was not up to the task?

That's what I do now, I run SA from my i7 920 desktop.

[najames]

Thanks guys!!

I thought I should be able to map a drive over and have it work, wanted to see if others have been doing it this way. The only thing that concerns me is if the wife records something and I'm in the middle of doing it when she wants to watch it.

Ah yes Stanger, but you are part of my problem!! I read about the old Athlon XP (?) based system you had and thought, "Hmmm, if he can run Sage with that thing I should be able to do it with a dual core, should even be able to run each core in a seperate VM since I'd have a much faster core".

Uggg, I knew about Intel's VT-x, didn't pay attention about Intel's "Virtualization Technology for Directed I/O" (VT-d). AMD has a different name for about the same thing it looks like, don't know if it works with Sage like Intel's VD (hehehehe) stuff. http://en.wikipedia.org/wiki/IOMMU

Those evil reviews of the 18W dual core Atoms and 25W i3/5 were what got me looking at my 90W idle. It also looks like a 620 series AMD quad undervolted would take about 45W idle and 90W load, extra cores for a VM are good too, sprinkle lots of RAM too. I am glad to read you did it, helps me know it is definately possible.

Reading about the server recommendation here got me thinking of AMD vs Intel.
http://www.avsforum.com/avs-vb/showt...18816470#Media Storage Server

I hacked around in WHS power management last night. I set it to shut down stuff in 15 min of non-use. My wife was watching recordings off an extender today and she said nothing appeared abnormal when she started it up. I will check the power usage while it is in sleep mode or whatever the name is.

I am getting really tired of researching and want to start the building/testing phase soon. I would have to have a complete well integrated system or it would be a failure.

Glad to see someone else in my boat, misery loves company.
http://hardforum.com/showthread.php?t=1532520

[stanger89]

Originally Posted by najames

Thanks guys!!

I thought I should be able to map a drive over and have it work, wanted to see if others have been doing it this way. The only thing that concerns me is if the wife records something and I'm in the middle of doing it when she wants to watch it.

Not sure what you mean.

Ah yes Stanger, but you are part of my problem!! I read about the old Athlon XP (?) based system you had and thought, "Hmmm, if he can run Sage with that thing I should be able to do it with a dual core, should even be able to run each core in a seperate VM since I'd have a much faster core".

Uggg, I knew about Intel's VT-x, didn't pay attention about Intel's "Virtualization Technology for Directed I/O" (VT-d). AMD has a different name for about the same thing it looks like, don't know if it works with Sage like Intel's VD (hehehehe) stuff. http://en.wikipedia.org/wiki/IOMMU

Unfortunately I've not been able to find anything that actually has IOMMU on it.

Those evil reviews of the 18W dual core Atoms and 25W i3/5 were what got me looking at my 90W idle. It also looks like a 620 series AMD quad undervolted would take about 45W idle and 90W load, extra cores for a VM are good too, sprinkle lots of RAM too. I am glad to read you did it, helps me know it is definately possible.

Just remember when you're comparing your 90W to their 25W, all the extra stuff you've got. You've probably got a couple more HDDs (~8W/drive) tuners, etc. The actual delta is probably smaller than you'd expect.

[paulbeers]

Originally Posted by stanger89

Just remember when you're comparing your 90W to their 25W, all the extra stuff you've got. You've probably got a couple more HDDs (~8W/drive) tuners, etc. The actual delta is probably smaller than you'd expect.

I have to agree with Stanger here (again as usual). My old Sage server had an AMD x2 4850e which is/was a lower power processor. While sufficient for everything (except for on the fly transcoding of H.264 HDPVR recordings), it also meant that comskip took longer to complete and meant I could really only run 1 video thru at a time. I was very concerned about the increase in power usage of my Sage server, but I finally decided to upgrade to the AMD x4 630 and truthfully my system rarely uses any more power than my old one. In fact it is really a push because while it can consume more power under full load, it spends a lot more time at idle so I really believe its about equal. Further even under full load I have only noticed about a 30-40 watt more power usage which, when you are running 9 hard drives, an optical drive, etc. like I do, is really only a 20-25% increase in power (again the additional time spent at idle/lowest power level offsets the increase).

Just my .02 worth.

[najames]

I did yard work, took the Mrs. out for lunch, ran some errands, watched the first half of a preseason NFL game, and the rest of the time has been researching this virtual crap. It took a couple Aleeves (sp?) in the middle to wade through it all. A real, not virtual headache.

AMD was supposed to have IOMMU working on the 890FX chipset, but evidently it doesn't work and motherboard manufacturers don't seem to care about it. There are 3 server G34 chipsets have it, but CPU/board costs are too much for my needs. An 8 or 12 core "Mangy" CPU does sound fun though!!

This is a list of desktop Intel CPUs that have VT-d enabled, at least I could find information on Intel's stuff. AMD is lacking in specs/documentation, and I even like AMD too. I hope my link works ok. http://ark.intel.com/MySearch.aspx?s...rketSegment=DT

I did run VMware Go on this AMD x2 on a 780G motherboard box and it said that hardware was unknown, but would try to install vSphere. I didn't try it.

I also spent time reading more about VirtualBox and the new VMWare player that lets you create VMs. VMware Player 3.1 looks like it can deal with hardware pretty good too. I found that VirtualBox could be installed on Win7 (Vail too?) and FreeNAS (ZFS) could be run virtually, likely UNraid also. http://www.sysprobs.com/freenas-iscs...ows-7-fun-test

Maybe you're both might be right about 90W usage on this box "not being too bad". I have 4 drives and a DVD burner I could take out and use a USB version instead. I agree, in their i3 tests they are only running one drive, sometimes a 2.5" laptop drive.

If I stick with this PC, I still need to consider WHS Vail, Sage 7, and UNraid or FreeNAS or something perhaps. I might do it with one of the other VM technologies on the big i7 920.

[stanger89]

Originally Posted by najames

I did yard work, took the Mrs. out for lunch, ran some errands, watched the first half of a preseason NFL game, and the rest of the time has been researching this virtual crap. It took a couple Aleeves (sp?) in the middle to wade through it all. A real, not virtual headache.

AMD was supposed to have IOMMU working on the 890FX chipset, but evidently it doesn't work and motherboard manufacturers don't seem to care about it. There are 3 server G34 chipsets have it, but CPU/board costs are too much for my needs. An 8 or 12 core "Mangy" CPU does sound fun though!!

This is a list of desktop Intel CPUs that have VT-d enabled, at least I could find information on Intel's stuff. AMD is lacking in specs/documentation, and I even like AMD too. I hope my link works ok. http://ark.intel.com/MySearch.aspx?s...rketSegment=DT

The DQ57TM supports VT-d too, and that works with the i5's.

I did run VMware Go on this AMD x2 on a 780G motherboard box and it said that hardware was unknown, but would try to install vSphere. I didn't try it.

ESXi 4.x runs just fine on a 780G. But you won't get PCI Passthrough. I'm not sure what you installed with VMware Go because AFAIK that's not actually a "product", more of a way of installing/managing other products (like ESXi).

Maybe you're both might be right about 90W usage on this box "not being too bad". I have 4 drives and a DVD burner I could take out and use a USB version instead. I agree, in their i3 tests they are only running one drive, sometimes a 2.5" laptop drive.

Let's just put things in perspective, my Athlon XP server right now is about 300W.

If I stick with this PC, I still need to consider WHS Vail, Sage 7, and UNraid or FreeNAS or something perhaps. I might do it with one of the other VM technologies on the big i7 920.

Most likely VT-d is supported on your i7, it works on mine, which is in a DX58SO board.

FWIW I'd just stick with a hypervisor (ESXi) rather than VM software running on an OS (Player). Just seems like performance should be better that way with less things running on top of each other.

[najames]

Stanger, do you know if the DQ57TM have any voltage adjustments in the BIOS? I doubt it in Intel "business" boards.

I didn't install anything else with VMware Go. They claimed it would scan your systems and tell you which was best to use, etc. I wanted to see what it said about installing on the 780 board, didn't test the Gigabyte EX58-UD4P i7 board.

My Athlon XP server right now is about 300W <== EYE CARUMBA!!

Are you sure about the VT-d on the DX58SO? I don't think it is working on mine. http://ark.intel.com/Product.aspx?id=36888

I downloaded Vail, FreeNAS, VirtualBox, and VMWare Player today. Installed the two virtual pieces on Win7 Pro and played around with VMWare mostly but couldn't get the LAN working in the FreeNAS VM for some reason. No LAN means no browser interface in to make drive setups. The day was a

I did measure the i7 920 setup today (2 drives, 12GB RAM), it was about 111W at idle.

[stanger89]

Originally Posted by najames

Stanger, do you know if the DQ57TM have any voltage adjustments in the BIOS? I doubt it in Intel "business" boards.

No I don't but then again, it doesn't really matter IMO. If you're building a server, number one priority is stability IMO. So I wouldn't want to be messing with voltages.

I didn't install anything else with VMware Go. They claimed it would scan your systems and tell you which was best to use, etc. I wanted to see what it said about installing on the 780 board, didn't test the Gigabyte EX58-UD4P i7 board.

My Athlon XP server right now is about 300W <== EYE CARUMBA!!

Are you sure about the VT-d on the DX58SO? I don't think it is working on mine. http://ark.intel.com/Product.aspx?id=36888

I know VMDirectPath is enabled. ESXi let me configure PCI devices for passthrough.

[najames]

My Intel server and AMD PC have been undervolted since day one without any problems. Server has been on 24/7 since June 2009, often goes several months between reboots. I've been running them like this for years. Little Biostar TA690 AMD x2 rig I had ran undervolted and overclocked for 2 years. No matter how crazy the settings it just happily chugged along. I wish I had it for a server now, sent it to a computerless nephew.

I think part of the problem I'm having is no confidence if a board or CPU will actually be VMDirectPath (or AMDs version) enabled. I've spent so much time researching that I'm about ready to punt and just build a low power rig, I can always make something work.

Do you ever go to the drag strip in Cedar Falls Stanger? I used to go there a lot eons ago.

[stanger89]

Originally Posted by najames

My Intel server and AMD PC have been undervolted since day one without any problems. Server has been on 24/7 since June 2009, often goes several months between reboots. I've been running them like this for years. Little Biostar TA690 AMD x2 rig I had ran undervolted and overclocked for 2 years. No matter how crazy the settings it just happily chugged along. I wish I had it for a server now, sent it to a computerless nephew.

Not saying you can't, I'm just saying I wouldn't. It's not something I care to play with so I haven't investigated the functionality.

[antplugger]

+1 for ESXi band wagon. I have been running a proof of concept for a couple of months now and its been perfect.

Server BOM
DQ45CB - Purchased it for VT-d support
CPU - Currently a tiny Core Duo 2300
4GB Memory
4 x 1TB Disks - RAID 5 and RAID 1
4 x 400GB - RAID 0 for backup
Adaptec 3805 Raid Contoller
1 x Gig Ethernet
HDHR
Will get a HD-PVR soon.

Running 3 VM's
1) OpenSuse - as NAS (AFP and SMB, TimeMachine backup)
2) XP for SageTV and Com Skip
3) XP for Girder, YAC, Backup software, Air video etc

Firstly let me say for something with such a tiny this thing runs fantastically - can record to HD shows and watch back no problems while doing a backup from my Mac. I think the key is the Disk controller - The adaptec 3805 cost as much as the PC but its performance has made a real difference to how well the VM's run and increase concurrency.

There is a known gotcha - VT-d does not work with the 3805. So you cant use PCI passthru (I wanted to use 2200's) so tyou have to use USB or network capture devices.

Cheers

[najames]

antplugger, you da big dog!!

Today I found out today that Q45s support VT-d too, also a BIOS fix got several of the the new 890FX AMD boards going with Xen.

http://wiki.xensource.com/xenwiki/VTdHowTo

I gotta go, but will have some questions for you real soon. Hmm, what the heck is a 2300? I don't need that many disks either, hopefully not a RAID controller.

Forgot, you are right Stanger, I read that my Gigabyte i7 920 board also has a BIOS that enables VT-d. Confusing, i5 650/660/670 seems to be CPU based VT-d specs, other stuff is chipset based.

[antplugger]

Sorry a typo - the CPU is a INTEL 6300

[najames]

Thanks Antplugger, I'm looking for a guage to go by. Are you using the first one here?

Intel Core 2 Duo E6300 Conroe 1.86GHz 2M shared L2 Cache LGA 775 65W Dual

Intel Pentium E6300 Wolfdale 2.8GHz 2MB L2 Cache LGA 775 65W

[harrijay]

+1 for runing ESXi with no issues.

Server is running an Intel Xeon x3440 probably a little overkill but I had one window to get a server and I decided to get something I can be happy with for many years.

I have been using VmDirectPath to pass through my A180 PCI HD tuner to a linux host without any problems. I am also passing 1.5 TB drive directly to the linux host also using RDM.

I have a Windows XP VM that does all of my comskipping and serving up playon content without any issues also

Jayson

[antplugger]

It's the first one you listed. The first core duo I ever purchased. I will be moving over a q9550 from another server when I have things pretty much settled down on esxi.

[najames]

Originally Posted by antplugger

It's the first one you listed. The first core duo I ever purchased.

Wow Antplugger, that is a little CPU if you can run Sage and Suse on it, plus another WinXP VM.

Jayson, what motherboard hardware are you running? Are you running it 24/7 as your main box?

These examples are exactly the types of things I want to have my server do. I spent my lunch hour and part of the afternoon while programs were running reading about the vague specifications for ESXi and Xen.

I spent time researching power ratings of various systems too. They are using a substantial video card, but it gives a direct comparison of sorts. This link shows the Intel Q6600 I have now, an AMD 620, Intel i5 661 which should be similar to the 650/660/670. I'm still not entirely sure of what board is required since the CPU has a VT-d "chipset" built in.

http://www.techspot.com/review/233-i...61/page15.html

Another with some of the AMD dual cores. I'd have to get an 890FX board and perhaps use a dinky low power ATI Rage type video card with AMD.

http://www.techspot.com/review/273-b...up/page10.html

Another option is getting a cheap Dell (or similar) server with the single Xeon simialr to what Jayson has, but I really prefer building my own box.

[harrijay]

I am running an Intel S3420PLX

I keep this running 24/7 but i do not use the server or VM for web browsing or day to day use. I have thought about trying to passthrough a video cared to a Windows VM to see how that would work but have not gotten around to it. The linux host that runs sage serves up a web server, dhcp, dns, and windows file sharing, and have not seen any kind of a hiccup related to be virutalized.

Jayson