SEEKER EXCEPTION THROWN:java.lang.OutOfMemoryError

[Flips]

I'm getting desperate here. I'm hoping someone out there can help.

Every few days (not exactly 3, sometimes 1 sometimes 5) when I get home from work I turn on the TV (usually with a hot plate of dinner in my hands) and it says that the Sage client cannot connect to the service. I have the same message on the server. The service on the server won't respond to a start or a stop.

The error log, at the point of when it starts, is below. The last 7 lines repeat over and over. My heap is set to 512. The service doesn't seem to be using a lot of memory. Which is reflected in the first line of the error log I posted.

I restart the server and it's fine, for a few days. I replaced one of the recording drives because I had a couple of bad blocks on it over the weekend, but that didn't seem to help.

I have two Fusion 3's and 3 PVR-250's. I've done some memory tests, and I'm running Prime95 overnight tonight.

I contacted support and they didn't have any answers for me. They said that they didn't really know what it could be. They said it might be an issue with the PVR-250, and I should update my driver. Well I'm using the latest one which is from 2005

Here's my java version info:
java version "1.6.0_12"
Java(TM) SE Runtime Environment (build 1.6.0_12-b04)
Java HotSpot(TM) Client VM (build 11.2-b01, mixed mode, sharing)

Anyone have any suggestions? I'm at wits end here.

Fri 3/27 9:35:05.734 MemStats: Used=75MB Total=94MB Max=532MB
Fri 3/27 9:35:05.734 MARK 1 currRecord=null enc=FusionHDTV, WDM Video Capture #2 clients=[] ir=false
Fri 3/27 9:35:05.735 Seeker in AUTOMATIC mode nextRecord=A[5059625,4649895,"VeggieTales",35520@0327.14:00,30] nextTTA=15894266
Fri 3/27 9:35:05.735 newRecord=null
Fri 3/27 9:35:05.735 NOTHING TO RECORD FOR NOW...
Fri 3/27 9:35:05.735 MARK 1 currRecord=null enc=Hauppauge WinTV PVR PCI II Capture #3 clients=[] ir=false
Fri 3/27 9:35:05.735 Seeker in AUTOMATIC mode nextRecord=A[5048609,5048593,"Dog Whisperer",24959@0327.13:00,60] nextTTA=12294266
Fri 3/27 9:35:05.735 newRecord=null
Fri 3/27 9:35:05.735 NOTHING TO RECORD FOR NOW...
Fri 3/27 9:35:05.736 MARK 1 currRecord=null enc=Hauppauge WinTV PVR PCI II Capture clients=[] ir=false
Fri 3/27 9:35:05.736 Seeker in AUTOMATIC mode nextRecord=null nextTTA=9223372036854775807
Fri 3/27 9:35:05.736 newRecord=null
Fri 3/27 9:35:05.736 NOTHING TO RECORD FOR NOW...
Fri 3/27 9:35:05.736 MARK 1 currRecord=null enc=FusionHDTV, WDM Video Capture clients=[] ir=false
Fri 3/27 9:35:05.736 Seeker in AUTOMATIC mode nextRecord=null nextTTA=9223372036854775807
Fri 3/27 9:35:05.736 newRecord=null
Fri 3/27 9:35:05.736 NOTHING TO RECORD FOR NOW...
Fri 3/27 9:35:05.751 SEEKER EXCEPTION THROWN:java.lang.OutOfMemoryError: unable to create new native thread
Fri 3/27 9:35:05.786 java.lang.Thread.start0(Native Method)
Fri 3/27 9:35:05.786 java.lang.Thread.start(Unknown Source)
Fri 3/27 9:35:05.786 sage.bp.hN(Unknown Source)
Fri 3/27 9:35:05.786 sage.ca.kL(Unknown Source)
Fri 3/27 9:35:05.786 sage.ca.run(Unknown Source)
Fri 3/27 9:35:05.786 java.lang.Thread.run(Unknown Source)
Fri 3/27 9:35:05.787 SEEKER EXCEPTION THROWN:java.lang.OutOfMemoryError: unable to create new native thread
Fri 3/27 9:35:05.799 java.lang.Thread.start0(Native Method)
Fri 3/27 9:35:05.799 java.lang.Thread.start(Unknown Source)
Fri 3/27 9:35:05.799 sage.bp.hN(Unknown Source)
Fri 3/27 9:35:05.800 sage.ca.kL(Unknown Source)
Fri 3/27 9:35:05.800 sage.ca.run(Unknown Source)
Fri 3/27 9:35:05.800 java.lang.Thread.run(Unknown Source)

[MrD]

Has this config ever worked satisfactorily?

Are these PCI slot cards, implying 5 pci slots on your motherboard?

Are you sure every slot is bus master or is one of them a slave slot?

The 250s are bus master devices, not sure about the Fusion, but I believe that it is.

What is the MB you are using?

[Flips]

I've got 5 PCI slots all filled with cards. My motherboard is a MSI K8N Neo2 Platinum.
http://www.msicomputer.com/product/p..._neo2_platinum

According to the description all 5 are master PCI. I have never known about PCI master vs. slave. I will have to keep that in mind in the future.

The system was very stable for a number of years. Then all at about the same time I did the following:
Move the server to a rack in the basement and build a client
Reinstall windows/sage due to a failed system drive
Add a fusion and 2 more PVR-250 cards.

As you can see I wasn't really taking baby steps in the changes.

Could this be a power supply problem? I need to check what what wattage in have in there, but it is probably 400-450. I would think I would be having hard crashes if that were the case instead of these java errors.

I attached my IRQ configuration, anything look like an issue there?

I ran P95 for 9 hours, no warnings or errors.

[paulbeers]

You are running java 1.6_12. There's your first problem. That is a horrible version of java. You know its bad when 13 was released not soon after this one to fix 12. Upgrade to 6u13 and see if that fixes your problem.

[Flips]

Thanks pb. I will try that tonight. A few weeks ago I cleaned off all the old versions and updated to the latest. My timing must have been bad to get v12.

I will definitely do this but I'm not sure it will help as I was having this issue with older java versions as well.

[stuckless]

I don't know much about your issue, but here's a PSU/Watt calculator, that may be of interesting if you are looking to estimate how much power your system requires....

http://www.journeysystems.com/?power_supply_calculator

[Flips]

okay... so things are getting stranger now... Here what's I got when I got home today. I had the spinning circle of death on the client seemed to be an issue with the photo slideshow I have on another computer. When I would try to play channel live I would get:

There was an error playing () ERROR (-17): The playback request could not be completed due to a network failure.

For a media file I would get:
Files selected for playback are no longer on disk.

Although Sage was at least responding unlike before. Here's a snip of my error log: The ministry/carny errors repeated over and over again since about 10:00 last night.

Tue 4/7 16:07:58.383 Carny waiting for awhile...
Tue 4/7 16:10:15.388 Ministry is waiting for 180 sec
Tue 4/7 16:13:15.396 Ministry is waiting for 180 sec
Tue 4/7 16:16:15.404 Ministry is waiting for 180 sec
Tue 4/7 16:19:15.306 Ministry is waiting for 180 sec
Tue 4/7 16:22:15.314 Ministry is waiting for 180 sec
Tue 4/7 16:25:15.323 Ministry is waiting for 180 sec
Tue 4/7 16:28:15.331 Ministry is waiting for 180 sec
Tue 4/7 16:31:15.339 Ministry is waiting for 180 sec
Tue 4/7 16:34:15.348 Ministry is waiting for 180 sec
Tue 4/7 16:37:15.356 Ministry is waiting for 180 sec
Tue 4/7 16:40:15.364 Ministry is waiting for 180 sec
Tue 4/7 16:43:15.373 Ministry is waiting for 180 sec
Tue 4/7 16:46:15.381 Ministry is waiting for 180 sec
Tue 4/7 16:49:15.390 Ministry is waiting for 180 sec
Tue 4/7 16:52:15.398 Ministry is waiting for 180 sec
Tue 4/7 16:55:15.406 Ministry is waiting for 180 sec
Tue 4/7 16:58:15.309 Ministry is waiting for 180 sec
Tue 4/7 17:01:15.317 Ministry is waiting for 180 sec
Tue 4/7 17:04:15.326 Ministry is waiting for 180 sec
Tue 4/7 17:07:15.334 Ministry is waiting for 180 sec
Tue 4/7 17:07:58.338 Carny waiting for awhile...
Tue 4/7 17:10:15.342 Ministry is waiting for 180 sec
Tue 4/7 17:13:15.351 Ministry is waiting for 180 sec
Tue 4/7 17:16:15.359 Ministry is waiting for 180 sec
Tue 4/7 17:19:15.367 Ministry is waiting for 180 sec
Tue 4/7 17:22:15.376 Ministry is waiting for 180 sec
Tue 4/7 17:25:15.384 Ministry is waiting for 180 sec
Tue 4/7 17:28:15.393 Ministry is waiting for 180 sec
Tue 4/7 17:31:15.401 Ministry is waiting for 180 sec
Tue 4/7 17:34:15.306 Ministry is waiting for 180 sec
Tue 4/7 17:37:15.315 Ministry is waiting for 180 sec
Tue 4/7 17:40:15.323 Ministry is waiting for 180 sec
Tue 4/7 17:43:15.332 Ministry is waiting for 180 sec
Tue 4/7 17:46:15.348 Ministry is waiting for 180 sec
Tue 4/7 17:48:23.136 Seeker.finishWatch(/127.0.0.1:1044@@SAGETV_PROCESS_LOCAL_UI)
Tue 4/7 17:49:15.356 Ministry is waiting for 180 sec
Tue 4/7 17:49:45.911 Error communicating with server:java.io.EOFException
Tue 4/7 17:49:46.292 Cleaning up c/s connection
Tue 4/7 17:49:46.346 NetworkManager CommunicationFailure : /127.0.0.1:1044 type=1
Tue 4/7 17:49:46.357 NetworkClient fullCleanup /127.0.0.1:1044
Tue 4/7 17:49:46.451 Cleaning up c/s connection
Tue 4/7 17:49:46.603 Cleaning up c/s connection
Tue 4/7 17:49:47.381 Error with c/s comm:java.lang.NullPointerException
Tue 4/7 17:49:47.381 NetworkManager CommunicationFailure : /127.0.0.1:1044 type=2
Tue 4/7 17:49:47.381 NetworkManager CommunicationFailure : /127.0.0.1:1044 type=2
Tue 4/7 17:49:49.992 Error communicating with server:java.lang.NullPointerException
Tue 4/7 17:49:49.997 Cleaning up c/s connection
Tue 4/7 17:49:49.997 NetworkManager CommunicationFailure : /127.0.0.1:1044 type=1
Tue 4/7 17:49:49.998 MsgSend thread terminating for /127.0.0.1:1044
Tue 4/7 17:49:54.477 SageTV received connection from:Socket[addr=/127.0.0.1,port=2165,localport=42024]
Tue 4/7 17:49:56.216 Sending DB to client of size:15681788
Tue 4/7 17:50:05.994 SageTV received connection from:Socket[addr=/127.0.0.1,port=2166,localport=42024]
Tue 4/7 17:50:06.178 Error w/SageTV client connection:java.lang.OutOfMemoryError: unable to create new native thread
Tue 4/7 17:50:06.179 Cleaning up c/s connection
Tue 4/7 17:50:06.313 Scheduler awoken
Tue 4/7 17:52:15.365 Ministry is waiting for 180 sec
Tue 4/7 17:55:15.373 Ministry is waiting for 180 sec
Tue 4/7 17:58:15.382 Ministry is waiting for 180 sec
Tue 4/7 17:58:17.027 Error with c/s comm:java.io.EOFException
Tue 4/7 17:58:17.028 NetworkManager CommunicationFailure : /127.0.0.1:2165 type=2
Tue 4/7 17:58:17.028 NetworkClient fullCleanup /127.0.0.1:2165
Tue 4/7 17:58:17.028 Cleaning up c/s connection
Tue 4/7 17:58:17.073 Cleaning up c/s connection

Anyone have any ideas? Could it be a bad network card on the mobo maybe?

I did that power supply calculator and I was getting 275 watts before the cards? Any idea what each of the tuner cards might pull? If they all pulled 25 watts, I would be up against the 400 watt limit. Do you think that might be possible?

[paulbeers]

Did you delete all the java versions off your server and install 6u13? This sounds exactly like the issues I was having when my computer inadvertently installed 6u12. It completely hoses Java. You have to completely uninstall all versions of java (with Sage stopped) and reinstall.

[MrD]

Originally Posted by Flips

I've got 5 PCI slots all filled with cards. My motherboard is a MSI K8N Neo2 Platinum.
http://www.msicomputer.com/product/p..._neo2_platinum

According to the description all 5 are master PCI. I have never known about PCI master vs. slave. I will have to keep that in mind in the future.

The system was very stable for a number of years. Then all at about the same time I did the following:
Move the server to a rack in the basement and build a client
Reinstall windows/sage due to a failed system drive
Add a fusion and 2 more PVR-250 cards.

As you can see I wasn't really taking baby steps in the changes.

Could this be a power supply problem? I need to check what what wattage in have in there, but it is probably 400-450. I would think I would be having hard crashes if that were the case instead of these java errors.

I attached my IRQ configuration, anything look like an issue there?

I ran P95 for 9 hours, no warnings or errors.

I have this board... it's not being used It was very sensitive to power supplies (i.e. it would not post with certain power supplies).

Page 2-25 of the manual shows that slot 2 and slot 5 have the same interrupt line. Meaning the PCI card drivers need to be able to share interrupts. Not sure if the Hauppauge / Fusion card drivers are that sophisticated.

What cards do you have in slot 2 and 5? I would try re-orienting the cards or pulling a card out of slot 5... plus it's orange

A slave slot does not have the ability to control the PCI bus (i.e. the card would not be able to initiate a bus transfer). It does not appear to be an issue though.

I would consider slot 2 / 5 sharing the hardware line to be a place to start.

[Flips]

Originally Posted by paulbeers

Did you delete all the java versions off your server and install 6u13? This sounds exactly like the issues I was having when my computer inadvertently installed 6u12. It completely hoses Java. You have to completely uninstall all versions of java (with Sage stopped) and reinstall.

pb, I got this error before I did the java update last night. I didn't do an uninstall first so this morning I uninstalled and reinstalled. Glad to hear that you have experienced and fixed this one. Maybe this was unrelated to the original issue and it was just an issue with 6u12. Maybe 13 will be my lucky update.

Originally Posted by MrD

I have this board... it's not being used It was very sensitive to power supplies (i.e. it would not post with certain power supplies).

Page 2-25 of the manual shows that slot 2 and slot 5 have the same interrupt line. Meaning the PCI card drivers need to be able to share interrupts. Not sure if the Hauppauge / Fusion card drivers are that sophisticated.

What cards do you have in slot 2 and 5? I would try re-orienting the cards or pulling a card out of slot 5... plus it's orange

A slave slot does not have the ability to control the PCI bus (i.e. the card would not be able to initiate a bus transfer). It does not appear to be an issue though.

I would consider slot 2 / 5 sharing the hardware line to be a place to start.

Interesting... I haven't had the same power supply issue that you mentioned, but good to know... I actually have 2 of these boards. My general desktop also uses this board (but I don't think it is the platinum version)

I was wondering about that orange slot, it just didn't look right. I think I will try to pull out the card and see what happens. I should be able to survive with only 2 PVR-250s..