SageTV Community  

Go Back   SageTV Community > Hardware Support > Hardware Support

Notices

Hardware Support Discussions related to using various hardware setups with SageTV products. Anything relating to capture cards, remotes, infrared receivers/transmitters, system compatibility or other hardware related problems or suggestions should be posted here.

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 01-28-2008, 10:06 PM
dmiraclejr dmiraclejr is offline
Sage User
 
Join Date: Sep 2006
Posts: 48
RAID, Quad cpu, or ? for improved server performance??

I often record 3 HD streams with 3 instances of comskip running, and playback 3 HD streams on extenders. For now 1 of the extenders is the HD extender and 2 are SD extenders (transcoding) but eventually all 3 extenders will be HD. I use the server in service mode and have the system described in my signature.

The problem is I often get intermittent skipping, pausing, audio drops, etc. Smarter people than I have diagnosed (in another thread) most likely that these problems are due to the massive amount of random seeks of the hard drives (two separate 3.0 Gbs disks that Sage allocates space). Right now I have limited comskip to one instance to try and deal with my problem but it still happens quite frequently.

My question is therefore:

Would RAID 0,1,5,6, or 10 help with my high hard drive random seeks/bandwidth needs and fix my problem? If so what RAID variation and how should I implement it the most cost efficiently?

Or would a Q6600 quad CPU fix it?

Or a combination of the two?


My WAF has been pretty low recently and I have considered buying a new or used lifetime Tivo for her since that is what we had before Sage and what she likes. I want to stick with Sage and have it just work.

Any guidance would be greatly appreciated

Thanks for all the help you forum members provide.
__________________
Server: C2D E6300 (oc to 3.0Ghz) / XP Pro SP3 / Gigabyte DS3 / 3GB Corsair DDR2 800 / BFG 7600GT/ VBox 150 tuner/ HDHomerun QAM / ATI 650 pro / AverTV M780 / USB-UIRT/ SageTV 6.4.8 / Java 1.6

Clients : 1 HD Extender, 2 wired MVPs and 1 Placeshifter
Reply With Quote
  #2  
Old 01-29-2008, 06:29 AM
BFisher BFisher is offline
Sage Aficionado
 
Join Date: Jul 2007
Posts: 394
I often record 2-3 streams (usually HD) and have 4 MVPs/1 HD Extender and haven't had any stuttering issues. I do have a Quad Core and it's rare my CPU % gets above 30-40% (when really working it hard).

I don't use RAID, not sure if it would help you or not. I assume you formatted your drives with large clusters? I have 3 700GB drives.

Sorry, nothing definitive (I'm not an expert), just giving you some food for thought.
Reply With Quote
  #3  
Old 01-29-2008, 07:45 AM
CollinR CollinR is offline
Sage Icon
 
Join Date: Dec 2004
Location: Tulsa, OK
Posts: 1,305
Next time you get stuttering bring up the task manager (Alt+Ctrl+Del) and see what your CPU usage is.


RAID is unlikely to help you, if this is a read/write issue. RAID arrays can allow you to read the disk faster but not so much on write to it.

64k cluster size?
Fragmented data?
Are these IDE drives?

Did you press F6 and load your SATA drivers before installing Windows?
Reply With Quote
  #4  
Old 01-29-2008, 07:52 AM
dmiraclejr dmiraclejr is offline
Sage User
 
Join Date: Sep 2006
Posts: 48
BFisher,

Thanks for your input. Do you also use comskip live while recording? How many instances?

Another note is that after I wrote that post last night I went to my system and started up 3 HD recordings (Leno, Letterman, Nightline) while watching the HD extender and letting the placeshifter be transcoded on the server, and 3 instances of comskip running. Guess what. Smooth as silk playback on the extender and placeshifter and the E6300 (oc @ 3.0 Ghz) was at 100% for about 5 minutes or so while comskip caught up with live tv (then settled to about 40%).

I don't understand it. I was kind of hoping that the server would choke so that I could feel good about being able to reproduce the problem and head in the right direction in fixing it. But it was smooth playback for an hour of watching.

Guess I will just let it go for a while and wait for the next mess up and go look at the logs. WAF be darned!
__________________
Server: C2D E6300 (oc to 3.0Ghz) / XP Pro SP3 / Gigabyte DS3 / 3GB Corsair DDR2 800 / BFG 7600GT/ VBox 150 tuner/ HDHomerun QAM / ATI 650 pro / AverTV M780 / USB-UIRT/ SageTV 6.4.8 / Java 1.6

Clients : 1 HD Extender, 2 wired MVPs and 1 Placeshifter
Reply With Quote
  #5  
Old 01-29-2008, 08:19 AM
dmiraclejr dmiraclejr is offline
Sage User
 
Join Date: Sep 2006
Posts: 48
Quote:
Originally Posted by CollinR View Post
Next time you get stuttering bring up the task manager (Alt+Ctrl+Del) and see what your CPU usage is.


RAID is unlikely to help you, if this is a read/write issue. RAID arrays can allow you to read the disk faster but not so much on write to it.

64k cluster size?
Fragmented data?
Are these IDE drives?

Did you press F6 and load your SATA drivers before installing Windows?
I have looked in the past and the CPU was 20-30% many times that I had problems.

Yes 64k clusters.

I never defragment recording drives. For that matter not the OS drive either. The server is dedicated to running the Sage service / comskip only.

No they are 3.0 Gbs SATA drives (one 400GB and one 500 GB Samsungs)

Never heard of press F6 to load SATA drivers before installing Windows before. What does that do?

Thanks for the ideas.

Also I have a gigabit network and I don't run antivirus or firewalls except for the one built into the router.
__________________
Server: C2D E6300 (oc to 3.0Ghz) / XP Pro SP3 / Gigabyte DS3 / 3GB Corsair DDR2 800 / BFG 7600GT/ VBox 150 tuner/ HDHomerun QAM / ATI 650 pro / AverTV M780 / USB-UIRT/ SageTV 6.4.8 / Java 1.6

Clients : 1 HD Extender, 2 wired MVPs and 1 Placeshifter
Reply With Quote
  #6  
Old 01-29-2008, 08:45 AM
rlvogel322's Avatar
rlvogel322 rlvogel322 is offline
Sage Advanced User
 
Join Date: Feb 2007
Location: Wisconsin
Posts: 163
Quote:
Originally Posted by dmiraclejr View Post
E6300 (oc @ 3.0 Ghz)
Wow! Nice work on that CPU. The fastest I ever got my E6300 was 2.8 Ghz on water.

Incidently, I also had the E6300 (oc @ 2.2 Ghz on air) running my WHS with SageTV and also acting as the client. No commercial skipping but recording 2 HD streams and playing back an HD file. It struggled at times.

Since I upgraded the server to a E6600 (oc @ 3.6 Ghz on air) I've had zero issues.

If you're going to upgrade, a Q6600 would be the way to go. Or you could wait until mid February and get a Q9450. Make sure your MB will support it though.

Also, defragment your recording drive once a week and that will help. If you haven't done it in a while, don't be surprised if it takes a while to complete. Pressing F6 during install is only for the raid/scsi drivers if you're going to run raid. You don't need to do that for Sata drives not running in Raid.

Last edited by rlvogel322; 01-29-2008 at 08:49 AM.
Reply With Quote
  #7  
Old 01-29-2008, 09:34 AM
CollinR CollinR is offline
Sage Icon
 
Join Date: Dec 2004
Location: Tulsa, OK
Posts: 1,305
Quote:
Originally Posted by dmiraclejr View Post
No they are 3.0 Gbs SATA drives (one 400GB and one 500 GB Samsungs)

Never heard of press F6 to load SATA drivers before installing Windows before. What does that do?

The only way to actually get that level of performance (3GB) is by installing the drivers for the controller when installing windows.

Use some software like sisoft sandra to detect the actual transfer capacity.
Reply With Quote
  #8  
Old 01-29-2008, 09:38 AM
BFisher BFisher is offline
Sage Aficionado
 
Join Date: Jul 2007
Posts: 394
Quote:
Originally Posted by dmiraclejr View Post
BFisher,

Thanks for your input. Do you also use comskip live while recording? How many instances?
I use ShowAnalyzer - and I allow up to 4 simultaneous instances - no issue.

I notice you don't defragment... I do. I have Diskeeper running - and have the time block set up so it will only defrag from 1am until 5am each day. It's rare I record during this period, and if I do it's probably only 1 show. And it's even rarer I watch at this time My drives are always nice and clean when I check their fragmentation.
Reply With Quote
  #9  
Old 01-29-2008, 07:08 PM
toricred's Avatar
toricred toricred is offline
Sage Icon
 
Join Date: Jan 2006
Location: Northern New Mexico
Posts: 1,729
Did I see that you also use MVPs? I've noticed that these really bring down my performance on my server. Your test situation didn't mention them so that might be the difference.
Reply With Quote
  #10  
Old 01-29-2008, 07:30 PM
evilpenguin's Avatar
evilpenguin evilpenguin is offline
SageTVaholic
 
Join Date: Aug 2003
Location: Seattle, WA
Posts: 3,696
Comskip tip

For running comskip on a very active system like that I recommend tuning the comskip.ini to give it a speed limit...

http://www.kaashoek.com/files/tuning.htm

Quote:
Reducing the CPU load
Comskip runs on “Idle” priority allowing other programs to execute normally. The Disk IO by Comskip can be rather high so it may be useful to reduce the throttle the execution speed of Comskip. The main parameters for this with their default values are

play_nice_sleep=10
play_nice_start=-1
play_nice_end=-1

The value of play_nice_sleep is the number of milliseconds Comskip will suspend execution after each frame processed. This will be done starting from play_nice_start till play_nice_end, both in 24 hour time format. From 7 in the morning till eleven in the evening will be

play_nice_start=700
play_nice_end=2300
I tuned mine so that a single instance of comskip can't run any faster than ~60fps. Its fast enough for live detection and, in the rare occasion it runs on a fully recorded file, it will go slow enough to prevent bandwidth issues.

Edit: You also never meantioned how many Hardrives vs. Recording sources you have. I try and keep mine at a 1:1 ratio so in the worst cases (5 recordings, 5 comskips, and a full client load) the bandwidth is spread over multiple drives.

Last edited by evilpenguin; 01-29-2008 at 07:35 PM.
Reply With Quote
  #11  
Old 01-29-2008, 08:23 PM
sandor's Avatar
sandor sandor is offline
Sage Expert
 
Join Date: Dec 2006
Location: Philadelphia, PA USA
Posts: 621
just as a data point, i like to maintain an equal number of recording drives as tuners (2 HDHR's = 4 unique recording drives) - i have found that this has kept me from ever having skipping/drop out issues in playback. I also have never defragged my drives, and always use 64k block for NTFS. Some of my recording drives are ext3 on a NASLite NAS system...


while SATA drives have high throughput, multiple IOs still kill standard hard drives (in other words, that max 70 MB/s is with one file, the more files accessed "simultaneously", the lower the combined throughput), 3.0 gbps is the theoretical bus limit, no single drive on today's consumer market will r/w more than about 100 MB/s (0.8 gbps))
__________________
MacBook Core2Duo 2 ghz
nVidia 9400M GPU
46" Sammy HLP4663 720p DLP
2x HDHR, all OTA
QNAP TS-809:
12.5 TB for Recordings/Imports/TimeMachine/Music
HD200 via 802.11n in Living Room
802.11n client in bedroom
Reply With Quote
  #12  
Old 01-30-2008, 05:48 AM
dmiraclejr dmiraclejr is offline
Sage User
 
Join Date: Sep 2006
Posts: 48
Lots of fix ideas. Thanks folks.

>>>I notice you don't defragment... I do. I have Diskeeper running - and have the time block set up so it will only defrag from 1am until 5am each day. It's rare I record during this period, and if I do it's probably only 1 show. And it's even rarer I watch at this time My drives are always nice and clean when I check their fragmentation.<<<
I used diskeeper on all my drives last night but the two recording drives still showed mostly fragmented (red). They are formated to 64k clusters.

>>>I tuned mine so that a single instance of comskip can't run any faster than ~60fps. Its fast enough for live detection and, in the rare occasion it runs on a fully recorded file, it will go slow enough to prevent bandwidth issues.<<<
What do the lines in the .ini file look like to calibrate conskip to 60fps?

>>>You also never meantioned how many Hardrives vs. Recording sources you have. I try and keep mine at a 1:1 ratio so in the worst cases (5 recordings, 5 comskips, and a full client load) the bandwidth is spread over multiple drives.<<<
Two recording hard drives, 4 HD OTA tuners (going to 6 with new HDHR), and 2 SD tuners.

>>>just as a data point, i like to maintain an equal number of recording drives as tuners (2 HDHR's = 4 unique recording drives) - i have found that this has kept me from ever having skipping/drop out issues in playback.<<<
Maybe I need 2 stick or 3 more hard drives in there of the 500 GB or 1TB variety! I guess yours are just allocated by Sage and not in a RAID configuration?
__________________
Server: C2D E6300 (oc to 3.0Ghz) / XP Pro SP3 / Gigabyte DS3 / 3GB Corsair DDR2 800 / BFG 7600GT/ VBox 150 tuner/ HDHomerun QAM / ATI 650 pro / AverTV M780 / USB-UIRT/ SageTV 6.4.8 / Java 1.6

Clients : 1 HD Extender, 2 wired MVPs and 1 Placeshifter
Reply With Quote
  #13  
Old 01-30-2008, 06:09 AM
dmiraclejr dmiraclejr is offline
Sage User
 
Join Date: Sep 2006
Posts: 48
I performed a little more experimentation last night and this is what I found.

Given:
3 HD tuners recording (ABC, CBS, NBC news)
1 SD tuner recording
watching only on the HD extender
3 instances of conskip running

I got bad stutter and/or macro blocking recorded into the recordings (and of coarse viewing). I have not tuned conskip to "play nice" yet nor had I defragged the hard drives yet. It had played smooth the other night under similar circumstances (Leno, Letterman, etc) but I can reliably reproduce the problem from 6:30pm-7:00pm every weeknight (I don't think it is just during this time period however).

The funny thing is that I see people post that they have 6 recordings or more going on due to soft padding and not have a problem. Or they have 3 HD recordings going with 3 instances of conskip while streaming to multiple extenders and still have now problems. I just don't get what I'm doing wrong. Very frustrating!

I guess my plan of action is going to be in this order:

Tune Conskip to 60 fps when I learn how
Record on defragged recording drives
Buy 2 (with total of 4 in system) new recording hard drives and let Sage allocate recordings
Put the 4 drives into some type of performance enhancing RAID
Buy a Q6600 cpu (eventually)

Notice the cheaper stuff is first.

Does this sound like a reasonable plan?
__________________
Server: C2D E6300 (oc to 3.0Ghz) / XP Pro SP3 / Gigabyte DS3 / 3GB Corsair DDR2 800 / BFG 7600GT/ VBox 150 tuner/ HDHomerun QAM / ATI 650 pro / AverTV M780 / USB-UIRT/ SageTV 6.4.8 / Java 1.6

Clients : 1 HD Extender, 2 wired MVPs and 1 Placeshifter
Reply With Quote
  #14  
Old 01-30-2008, 06:58 AM
kevine's Avatar
kevine kevine is offline
Sage Expert
 
Join Date: Aug 2007
Posts: 514
Quote:
Originally Posted by dmiraclejr View Post
I guess my plan of action is going to be in this order:

Tune Conskip to 60 fps when I learn how
Record on defragged recording drives
Buy 2 (with total of 4 in system) new recording hard drives and let Sage allocate recordings
Put the 4 drives into some type of performance enhancing RAID
Buy a Q6600 cpu (eventually)

Notice the cheaper stuff is first.

Does this sound like a reasonable plan?
Just make sure you do 1 at a time or you will never find out what was causing your problem. I have similar problems. Comskip is running overnight for me. I have 9 tuners and only 4 drives (All SATA and not RAID). My plan of action is JKDefrag daily and wait a week to see what kind of problems I have. Then I am going to start removing tuners 1 or 2 at a time each week. If/when (oh please when ) I find a sweet spot with this approach, I will start adding drives as I add tuners.

My problem is also 3 MVPs that could be all transcoding at the same time so as I do this, I will be checking task manager to see that my cpu is not pegged.
__________________
Server: Intel Core 2 Quad Q9550 (2.83GHz, 1333FSB), Asus P5B Deluxe MB w/Gb LAN, 4GB AData RAM, WHS, PVR-500 (NTSCx2), HDHomeRun x2 (4 QAM), HDPVR x2, 6.7 TB data pool, Comskip/SJQv3, HD200x3, SD-MVPx2
Reply With Quote
  #15  
Old 01-30-2008, 10:24 AM
rlvogel322's Avatar
rlvogel322 rlvogel322 is offline
Sage Advanced User
 
Join Date: Feb 2007
Location: Wisconsin
Posts: 163
Quote:
Originally Posted by dmiraclejr View Post
Buy a Q6600 cpu (eventually)
I don't know how comfortable you are with buying used stuff, but you can get a pretty good deal on Q6600 and even Q6700 through Ebay. I've done it in the past with good success.

I would think the Q6600 and Q6700 would be dropping siginificantly by the end of February. The new quads are set to hit the market then.

I'm not sure if it has been mentioned, but are you sure your drives are set to 3.0 Gbs transfer rate? The last two drives I purchased (OEM from the Egg) came with the jumper installed so that it was set to 1.5 Gbs. Just a thought...
Reply With Quote
  #16  
Old 01-30-2008, 01:18 PM
sandor's Avatar
sandor sandor is offline
Sage Expert
 
Join Date: Dec 2006
Location: Philadelphia, PA USA
Posts: 621
Quote:
Originally Posted by dmiraclejr View Post

>>>just as a data point, i like to maintain an equal number of recording drives as tuners (2 HDHR's = 4 unique recording drives) - i have found that this has kept me from ever having skipping/drop out issues in playback.<<<
Maybe I need 2 stick or 3 more hard drives in there of the 500 GB or 1TB variety! I guess yours are just allocated by Sage and not in a RAID configuration?

correct, i have (currently) 4 500 GB SATA drives used solely by Sage as recording drives, each is an independent drive, no RAID.

I'm slowly moving all my drives to my NAS ( http://www.serverelements.com ) so i currently have 2 NTFS drives locally on the Sage server, and 2 ext3 drives over gigabit enet on the NASLite box.

As another data point, the NASLite box can saturate gigabit enet with no problem
__________________
MacBook Core2Duo 2 ghz
nVidia 9400M GPU
46" Sammy HLP4663 720p DLP
2x HDHR, all OTA
QNAP TS-809:
12.5 TB for Recordings/Imports/TimeMachine/Music
HD200 via 802.11n in Living Room
802.11n client in bedroom
Reply With Quote
  #17  
Old 01-30-2008, 02:32 PM
BFisher BFisher is offline
Sage Aficionado
 
Join Date: Jul 2007
Posts: 394
not to hijack the thread... but what is the advantage of NAS devices versus adding more drives to the PC itself? Seems like you could run into network issues easier... (I'm assuming NAS device sits on the network, not directly connected to the PC)
Reply With Quote
  #18  
Old 01-31-2008, 12:35 AM
megamojo's Avatar
megamojo megamojo is offline
Sage Advanced User
 
Join Date: Apr 2006
Posts: 179
Quote:
Originally Posted by rlvogel322 View Post
I'm not sure if it has been mentioned, but are you sure your drives are set to 3.0 Gbs transfer rate? The last two drives I purchased (OEM from the Egg) came with the jumper installed so that it was set to 1.5 Gbs. Just a thought...
It's been alluded to before in this thread, but everyone needs to realize that the 1.5 Gb/s for SATA and 3 Gb/s for SATA II is the theoretical bandwidth of the interface, NOT the actual performace of the physical drives. Also, the b's are not capitalized, so it's gigabits, not gigabytes. Since there are 8 bits in a byte, 3 Gb/s is about 375 MB/s and 1.5 Gb/s is about 185 MB/s. (The 2 versions of SATA are also often called SATA 150 and SATA 300, meaning MB/s in this case.) You won't find any drive faster than 100 MB/s, and most will be around 70 MB/s or slower, so, playing around with jumpers and drivers and the like will get you absolutely nowhere since the physical speed of the drives is always the limiting factor.


Quote:
Originally Posted by sandor View Post
I'm slowly moving all my drives to my NAS ( http://www.serverelements.com ) so i currently have 2 NTFS drives locally on the Sage server, and 2 ext3 drives over gigabit enet on the NASLite box.

As another data point, the NASLite box can saturate gigabit enet with no problem
If you really want good performance from your NAS box, I would highly recommend using a different linux filesystem. Ext3 is a good general purpose filesystem, but does pretty badly with very large files, which is pretty much all you'll be using with Sage. XFS was specifically designed to work well with large files and it's been great for me. And for cryin' out loud, why are you paying $20 or $30 for that NAS software?! Grow up and make your own linux box, or at least user ReadyNAS (google it), which pretty much does everything NASLite does but it's free.

Edit: I meant freenas, not readynas, my mistake.

Last edited by megamojo; 01-31-2008 at 09:37 AM.
Reply With Quote
  #19  
Old 01-31-2008, 01:38 AM
evilpenguin's Avatar
evilpenguin evilpenguin is offline
SageTVaholic
 
Join Date: Aug 2003
Location: Seattle, WA
Posts: 3,696
Quote:
Originally Posted by megamojo View Post
You won't find any drive faster than 100 MB/s, and most will be around 70 MB/s or slower, so, playing around with jumpers and drivers and the like will get you absolutely nowhere since the physical speed of the drives is always the limiting factor.
I'm fairly sure that's untrue, setting your jumpers properly can by you 5-10MB extra throughput. Its not much, but in an HTPC setting that much extra can help a lot.
Reply With Quote
  #20  
Old 01-31-2008, 02:09 AM
megamojo's Avatar
megamojo megamojo is offline
Sage Advanced User
 
Join Date: Apr 2006
Posts: 179
Quote:
Originally Posted by evilpenguin View Post
I'm fairly sure that's untrue, setting your jumpers properly can by you 5-10MB extra throughput. Its not much, but in an HTPC setting that much extra can help a lot.
Agreed, but I'm pretty sure that 5-10 MB/s extra throughput is not the root cause of th OP's problem.
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Sage Server as a service in Win2003 drive mapping eguy SageTV Software 9 07-27-2007 01:10 PM
CPU use for MOBO Raid 1 Jesse Hardware Support 1 12-24-2006 10:34 AM
SageTV 4.1 is crashing Win2003 server SethV SageTV Software 7 02-03-2006 11:16 PM
Enabling Library share pegs Sage Server cpu mdmint SageTV Software 15 01-25-2005 02:59 AM
100% CPU for RC3 Server after Client Action plawlor SageTV Beta Test Software 3 05-10-2004 05:58 AM


All times are GMT -6. The time now is 06:40 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.