Uptime issues and Memory leaks...

[Eckwell]

Please see my update about this issue on another thread. I am running stable now for two days and have not seen the circle of death via the extender once. Client running much smoother now too. I can even let my wife use the HD100 to play music again!

http://forums.sagetv.com/forums/show...2&postcount=10

Maybe some of my changes might help you all...

[raoul]

Just an update that my memory issue has gotten a lot better and I have gone from needing to reboot every couple of days to great performance with the computer up for over a week.

I thought I was on the latest Java, but the built-in updater only took me to Java 6 Update 7.

I run SageMC with the advanced miniguide plugin.

I made two changes to my system that I can think of. I moved the commercial cutting software (comskip and dirmon) to another computer on my network to reduce the load on my sage server. The other thing that I changed is somehow my sageMC was setup to kick off comskip (old legacy feature that isn't really used) and I turned that off.

Not sure what else I could have done. I will report back if the problem returns or if I get some more information.

[taylormadearmy]

Originally Posted by taylormadearmy

I think I am seeing a similar issue.

Using SageMC plugin and advanced miniguide - everything else stock and 3 HDXs.

Running on WHS. Recently increased system ram from 1Gig to 2Gigs. Seems to have helped no end.

So far been up for 3 Days, 22 Hours, 25 Minutes, 34 Seconds... Which is about double the time it could have managed before without grinding to a halt. I guess time will tell if this is a definite fix for me or if I will start to run into issues again - just after a longer time.

How much memory are others here with problems running with?

T

My memory uprgade seems to have fixed the issue I was having. My box currently has an uptime of 11 Days, 5 Hours, 47 Minutes, 48 Seconds with no skips, jumps, stutters or swirly circles - would have been longer but a windows update restarted my box for me...

The SageTV service seems to hover around 560Mb for my box - so I guess that is too high a memory usage for 1Gb of Ram to be sufficient for without frequent page file swaps when other processes need memory?

Take Care
T

[DualQuad]

After starting this thread way back when I heeded various nuggets of advice and removed SageMC, cleaned up my client directory and low and behold my uptime issues went away. At this point my server had an uptime of 21+ days before something went screwy and required a reboot. This was a huge improvement. Extender responsiveness was vastly improved and memory usage remained low (200MB kind of range).

Thinking that I now had a foolproof solution, I built a lower powered server about a month and half ago (knocked 70W off the the original) and loaded up all of the latest and greatest versions of Sage, java and drivers. The uptime problems have returned. I bumped the max heap to 800MB and the in use memory for java seems to grow about 20MB a day. Heck, I even installed SageMC then disabled it in hopes some magic jar file gets installed. My machine is now at 330/419/799 MB after about a week of use as well as having poor extender performance.

I kicked log files on during the last reboot in anticipation of creating a trouble ticket.
Any other ideas to correct this or am I down to seeking Sage assistance?

Thanks in advance,
DQ

[Ian]

Thanks to user Nadim I pretty much fixed my issues (I have been going 3+ weeks now w/o reboot - used to crash every 3 days max!).

Check out this MS utility called Process Monitor:
http://technet.microsoft.com/en-us/s.../bb896645.aspx

The problems I was having were NOTcaused by SageTV or java...
The process monitor will show you everything that is going on process wise in Windows and the results of each process instruction (and what registry element they use etc.). At first I would NOT recommend running Sage while this is on if it is recording something because it wil show you all of the reads/writes that Sage performs as it records.

Though this Proc Mon app can filter output so you can focus on particular processes, I preferred to let it run for a bit unfiltered. It puts out a LOT of info in a short amount of time. Let it run for a little while and stop capturing and look at the recurring process usage by each app running on the PC.

Proc Mon showed me I had two apps that consistently ran all the time in the background, accessed registry a lot, and often had a return value of "registry info not found" (or something similar)for something they were doing. I figured if I had a memory leak, it was probably as a result of some process continually allocating memory but not getting a return value/freeing up the memory it was hoarding (the basic def. of a mem leak). So my approach was to focus on the apps that had return failures when accessing the registry.

The two apps for me were:
1) ArcSoft connect
2) Gigabyte power save monitor

The first one was loaded when I loaded the HD-PVR drivers. The second one was loaded with some SW I loaded when I built my PC with a GIGABYTE motherboard.

The second one was easy to get rid of, it had an uninstall feature. The first one is more problematic. ArcSoft does not make it easy to remove this thing. It is supposed to kind of keep track of what ArcSoft SW you have (Total Media Exterme for example) and make sure it is registered or check for updates. The way I removed it was similar to this:
http://www.computing.net/answers/win...ct/171029.html

(Google for "removing arcsoft connect" - lot of hits)

Hopefully this works for you - it worked for me. Since you just built this new PC there may be some SW that installed with your mobo or something that is eating your memory.

[DualQuad]

Ian, I will give it a try this weekend. From what little info I can gather, it seems that SageTv is blowing its heap. Maybe these weird apps are related/spawned by activity in Sage (like recordings) and the failure leaves something hanging in Sage.

Seems kinda far fetched, but I am out of ideas and appreciate the help.

Thanks,
DQ

[Ian]

Just some more info - I increased my java heap size a long time ago to 512 and it did nothing to help w/the crashing before I removed these backgd apps.

I am still on Sage 6.4.8 and java rev .11 (can'r remember exactly but it's a few months old). My server has 4GB of RAM.

My java mem usage grows to over 500M usage after a week or so, but hovers around 300/5xxM used. It goes back down sometimes. Despite all of this it runs fine and the HD100s are fast (all of my viewing is on HD100s).

One thing I notice about Sage (I'm using SageMC) is that you can really go from menu-to-menu repeatedly and it remembers all of these movements. From the remote you can go from home->guide->watching something then press TV or Video button, etc.
I suspect this eats up more of the java memory than say starting from the home menu, selecting something, then pressing the back button then selecting something else.

Sometimes I will pick up the remote after my kid has been using the TV and hit the "back" button repeatedly and go through about 10 different menus/guides before it stops going back. So it keeps all of this navigation in memory somewhere. Maybe this is where the java mem is going. It does seem to free it occasionally though, the number does not always increase.

Anyway hope you resolve you issue!

[DualQuad]

I have friends at work that are seeing the same kinds of issues. One of them is actually running two servers, one server has memory issues the other server runs seemingly forever with low usage. Its interesting that your machine memory seems to plateau, mine is stepping up everyday. There seems to be a burst of memory usage when an extender is started. On my fully functional machine I didn't see this much memory being used (though sage and java versions are now different)

I would think that the memory used for the screen "tree" would be freed when the extender shuts down. I can test that easy enough, maybe I will give it a go. I would think that the memory in use (first number of the three) would also eventually get down near to the level at which is indicated when the server is started.

I know other folks are seeing this and some have resorted to cron job restarts and the like. I was once one of the happy stable system guys and desperately seek to rejoin that group.

Thanks again,
DQ

[toricred]

I tried Process Monitor on my system this morning. I focussed on SageService and I'm getting a bunch of "NAME NOT FOUND" errors when trying to read or query two registry values. One is HKLM\Software\Policies\Microsoft\System\DnsClient the other is HKLM\Software\Policies\Microsoft\Windows NT\DnsClient. The values don't exist at all, but the DNSClient service is running. Any ideas?

[toricred]

OK, I had my DNS servers were configured backwards and the one that was listed as primary was dead. Now I've eliminated that problem, but Process Monitor is listing two other problems.

1. I am getting "FAST IO DISALLOWED" for some files while they aare recording sometimes. (I think this is related to the drive also being used to compress other files.

2. I am getting errors looking for the .mpgbuff files for some of the tuners.

Any suggestions for either of these?

[DualQuad]

I too am seeing a pile of Fast_IO_DISALLOWED dumping out during recordings.

DQ

[hugenbdd]

Same here, Fas_IO_DISALLOWED.

WHS new install, HD-PVR and HDHR.

I have removed the arcsoft services that was referenced earlier and have increased the java heap to 1024. I have also upgraded to Java update 12.

I have reverted to a script that restarts the sagetv service every night.

Originally Posted by DualQuad

I too am seeing a pile of Fast_IO_DISALLOWED dumping out during recordings.

DQ

Originally Posted by DualQuad

I too am seeing a pile of Fast_IO_DISALLOWED dumping out during recordings.

DQ

[toricred]

I just found the cause of this on my system. I was using the osdclock and osdplaytime plugins and they both had a bug that caused a memory leak. I'm now staying at 200mb in use with 4 HD extenders running for over 24 hours. In previous days this would have been well over 700mb. As I am the author of osdclock and the other plugin was based on my code I feel extremely guilty about this. I have now fixed it and updated the download section. I have included osdplaytime in the osdclock download because the author (gexi) doesn't seem to have been on the forums in about a year.

[Ian]

wow, very interesting! This is inline with the symptoms in my case as the system does not lose memory on mine unless you are using the front ends, and the more in use the worse it is. Can't wait to try the new d/l, thanks for the info!