SageTV Community  

Go Back   SageTV Community > SageTV Products > SageTV EPG Service

Notices

SageTV EPG Service Discussion related to the SageTV EPG Service used within SageTV. Questions about service area coverage, channel lineups, EPG listings, XMLTV, or anything else related to the service or programming guide data for SageTV should be posted here.

Reply
 
Thread Tools Search this Thread Display Modes
  #21  
Old 09-27-2018, 05:55 PM
RymCo RymCo is offline
Sage User
 
Join Date: Jul 2009
Location: Langton, Ontario
Posts: 10
Hiw did you end up fixing this? My guide is all No Data but I know I have an active SD subscription and I did find an error in the logs somewhere talking about NullPointer.

I thought maybe it was bad schedule and it would remedy itself but a month later and it hasn't.
Reply With Quote
  #22  
Old 09-30-2018, 07:06 AM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,112
I still have issues with No Data holes on my 1 machine (others are fine).
Unfortunately I haven't had much time to dig into it more.
The issue where wiz.bin would lock hasn't reoccurred (knock on wood).

What I do periodically (pain in the ***) is:
1. Stop Sage
2. delete the sdmd5prog & sdmd5sched files
3. edit sage.properties file removing the following lines:
epg_data_sources/<tuner number>/expanded_until=1538316079220
epg_data_sources/<tuner number>/last_run=1538229679219
epg_data_sources/<tuner number>/server_update_id=
4. Restart Sage and let it do a complete rebuild of the EPG data

This is usually good for a couple days then I start getting No Data holes again
Reply With Quote
  #23  
Old 09-30-2018, 08:25 AM
wnjj wnjj is online now
Sage Expert
 
Join Date: Jan 2009
Posts: 691
Quote:
Originally Posted by graywolf View Post
I still have issues with No Data holes on my 1 machine (others are fine).
Unfortunately I haven't had much time to dig into it more.
The issue where wiz.bin would lock hasn't reoccurred (knock on wood).

What I do periodically (pain in the ***) is:
1. Stop Sage
2. delete the sdmd5prog & sdmd5sched files
3. edit sage.properties file removing the following lines:
epg_data_sources/<tuner number>/expanded_until=1538316079220
epg_data_sources/<tuner number>/last_run=1538229679219
epg_data_sources/<tuner number>/server_update_id=
4. Restart Sage and let it do a complete rebuild of the EPG data

This is usually good for a couple days then I start getting No Data holes again
Are you running the null pointer fixed jar I shared? If that seems to fix at least the wiz.bin issue itís worth getting put into the main code base.
Reply With Quote
  #24  
Old 09-30-2018, 11:21 PM
JREkiwi's Avatar
JREkiwi JREkiwi is offline
Sage Icon
 
Join Date: Jan 2005
Location: Auckland, New Zealand
Posts: 2,014
Quote:
Originally Posted by wnjj View Post
Are you running the null pointer fixed jar I shared? If that seems to fix at least the wiz.bin issue it’s worth getting put into the main code base.
I've not had a single instance of the NullPointerException since running with the fixed jar and it was happening probably 4-5 times a week. Thanks

John
Reply With Quote
  #25  
Old 10-04-2018, 01:21 PM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,112
Yes. Been running the fix jar without issue
Iíd say it is ready to go into production
Reply With Quote
  #26  
Old 10-04-2018, 03:34 PM
wnjj wnjj is online now
Sage Expert
 
Join Date: Jan 2009
Posts: 691
Quote:
Originally Posted by graywolf View Post
Yes. Been running the fix jar without issue
Iíd say it is ready to go into production
That's good news. I'll see about submitting my changes to Github.
Reply With Quote
  #27  
Old 10-08-2018, 01:11 PM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,112
Happened again - NullPointerException

Wiz.bin.tmp = 10/6/2018 5:47 PM
Wiz.bak = 10/6/2018 5:46 PM

Code:
Sat 10/6 17:46:18.161 [EPG@16b5df0] Wizard compressing new file with version 87...
Sat 10/6 17:46:18.164 [EPG@16b5df0] Wizard writing out table info for Year
Sat 10/6 17:46:18.164 [EPG@16b5df0] Wizard writing out table info for Network
Sat 10/6 17:46:18.164 [EPG@16b5df0] Wizard writing out table info for Title
Sat 10/6 17:46:18.200 [EPG@16b5df0] Wizard writing out table info for Channel
Sat 10/6 17:46:18.202 [EPG@16b5df0] Wizard writing out table info for Bonus
Sat 10/6 17:46:18.206 [EPG@16b5df0] Wizard writing out table info for People
Sat 10/6 17:46:18.446 [EPG@16b5df0] Wizard writing out table info for SubCategory
Sat 10/6 17:46:18.447 [EPG@16b5df0] Wizard writing out table info for Rated
Sat 10/6 17:46:18.447 [EPG@16b5df0] Wizard writing out table info for ParentalRating
Sat 10/6 17:46:18.447 [EPG@16b5df0] Wizard writing out table info for ExtendedRating
Sat 10/6 17:46:18.447 [EPG@16b5df0] Wizard writing out table info for Category
Sat 10/6 17:46:18.447 [EPG@16b5df0] Wizard writing out table info for PrimeTitle
Sat 10/6 17:46:18.447 [EPG@16b5df0] Wizard writing out table info for Show
Sat 10/6 17:46:19.285 [EPG@16b5df0] Wizard writing out table info for Airing
Sat 10/6 17:46:19.720 [EPG@16b5df0] Wizard writing out table info for ManualRecord
Sat 10/6 17:46:19.735 [EPG@16b5df0] Wizard writing out table info for MediaFile
Sat 10/6 17:46:20.550 [EPG@16b5df0] Wizard writing out table info for Watched
Sat 10/6 17:46:20.562 [EPG@16b5df0] Wizard writing out table info for Agent
Sat 10/6 17:46:20.571 [EPG@16b5df0] Wizard writing out table info for Wasted
Sat 10/6 17:46:20.574 [EPG@16b5df0] Wizard writing out table info for Playlist
Sat 10/6 17:46:20.574 [EPG@16b5df0] Wizard writing out table info for TVEditorial
Sat 10/6 17:46:20.574 [EPG@16b5df0] Wizard writing out table info for SeriesInfo
Sat 10/6 17:46:20.811 [EPG@16b5df0] Exception during Maintenance:java.lang.NullPointerException
Sat 10/6 17:46:20.811 [EPG@16b5df0] java.lang.NullPointerException
Sat 10/6 17:46:20.811 [EPG@16b5df0] 	at sage.SeriesInfo.write(SeriesInfo.java:201)
Sat 10/6 17:46:20.811 [EPG@16b5df0] 	at sage.Wizard.saveDBFile(Wizard.java:7938)
Sat 10/6 17:46:20.811 [EPG@16b5df0] 	at sage.Wizard.maintenance(Wizard.java:2369)
Sat 10/6 17:46:20.811 [EPG@16b5df0] 	at sage.EPG.run(EPG.java:645)
Sat 10/6 17:46:20.811 [EPG@16b5df0] 	at java.lang.Thread.run(Unknown Source)
Sat 10/6 17:46:20.811 [EPG@16b5df0] sage.epg.sd.SDRipper@18e0de7 needs an update in 16:15:51
Sat 10/6 17:46:20.811 [EPG@16b5df0] EPG needs an update in 975 minutes
Sat 10/6 17:46:20.811 [EPG@16b5df0] EPG's works is done. Waiting...
Reply With Quote
  #28  
Old 10-08-2018, 04:45 PM
wnjj wnjj is online now
Sage Expert
 
Join Date: Jan 2009
Posts: 691
Well that sucks. Now that I looked over my "fixed" code yet again, I see another bug. Part-time code work is tough.

While it does not add any NULL people, the array length is still pre-determined when the array was originally defined. It's not sufficient to just filter the bad elements. The array needs to be the size of the "valid" amount of people. I need to loop through looking for valid people before creating the array, then fill it.

You should have at least had an "addSeriesInfo: Attempting to add null person!" warning message in your logs some time before this crash to help identify the source.

I'll see about (yet another) update for you when I get home.
Reply With Quote
  #29  
Old 10-08-2018, 09:16 PM
wnjj wnjj is online now
Sage Expert
 
Join Date: Jan 2009
Posts: 691
Newest code at https://www.dropbox.com/s/g3p5k0n16v...ix_100918?dl=0

Here's the latest code:
Code:
      if (people == null || people.length == 0)
      {
        series.people = Pooler.EMPTY_PERSON_ARRAY;
        series.characters = Pooler.EMPTY_STRING_ARRAY;
      }
      else
      {
        if (people.length != characters.length)
          throw new InternalError("ERROR length of people & character arrays do not match!");

        int sz = 0;
        for (int i = 0; i < people.length; i++)
        {
          if ((people[i] != null) && (characters[i] != null)) sz++;
        }
        series.people = new Person[sz];
        series.characters = new String[sz];
        for (int i = 0, j = 0; i < people.length; i++)
        {
          if ((people[i] != null) && (characters[i] != null))
          {
            series.people[j] = people[i];
            series.characters[j] = new String(characters[i]);
            j++;
          }
          else if (Sage.DBG) System.out.println("addSeriesInfo: Attempting to add null person or character!  Series=" + series.getTitle() + ", person #" + i);
        }
      }
Note: This jar also has a sortbyairingdate fix I was doing for another user. It shouldn't affect anything for this issue.

Last edited by wnjj; 10-09-2018 at 09:45 AM. Reason: Changed link to newer code than in this post.
Reply With Quote
  #30  
Old 10-09-2018, 07:56 AM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,112
Yes, found these:

Code:
Line 1359: Sat 10/6 17:33:51.674 [Timer-2875@17f4dee] addSeriesInfo: Attempting to add null person!  Series=The Dragon Prince, person #0
	Line 1360: Sat 10/6 17:33:51.674 [Timer-2875@17f4dee] addSeriesInfo: Attempting to add null person!  Series=The Dragon Prince, person #1
Will apply the new fix later today.

Greatly appreciated your looking into and working on the fix
Reply With Quote
  #31  
Old 10-09-2018, 08:39 AM
wnjj wnjj is online now
Sage Expert
 
Join Date: Jan 2009
Posts: 691
Quote:
Originally Posted by graywolf View Post
Yes, found these:

Code:
Line 1359: Sat 10/6 17:33:51.674 [Timer-2875@17f4dee] addSeriesInfo: Attempting to add null person!  Series=The Dragon Prince, person #0
	Line 1360: Sat 10/6 17:33:51.674 [Timer-2875@17f4dee] addSeriesInfo: Attempting to add null person!  Series=The Dragon Prince, person #1
Will apply the new fix later today.

Greatly appreciated your looking into and working on the fix
Thatís good info. Do you know where that show got its guide data? From SD?

Thanks for your patience with my hack job code.
Reply With Quote
  #32  
Old 10-09-2018, 09:43 AM
wnjj wnjj is online now
Sage Expert
 
Join Date: Jan 2009
Posts: 691
Good thing you said later today. I was thinking offline and realized there is a corner case that would probably still crash and yours may be that one. If the array of people has a non-zero size, but ALL of the people in it are NULL, the code will try to allocate an array of size 0. Additionally, the original Sage code never checked if the character array itself was NULL.

EDIT: As it turns out, Java allows arrays of size 0 which ironically is exactly what the Pooler.EMPTY_PERSON_ARRAY is so I guess my previous one would have worked. This one still is safer with regard to a NULL character array.

So one more round please.
Code:
      int sz = 0;
      if (people != null && characters != null)
      {
        if (people.length != characters.length)
          throw new InternalError("ERROR length of people & character arrays do not match!");

        for (int i = 0; i < people.length; i++)
        {
          if ((people[i] != null) && (characters[i] != null)) sz++;
        }
      }
      if (people == null || characters == null || sz == 0)
      {
        series.people = Pooler.EMPTY_PERSON_ARRAY;
        series.characters = Pooler.EMPTY_STRING_ARRAY;
      }
      else
      {
        series.people = new Person[sz];
        series.characters = new String[sz];
        for (int i = 0, j = 0; i < people.length; i++)
        {
          if ((people[i] != null) && (characters[i] != null))
          {
            series.people[j] = people[i];
            series.characters[j] = new String(characters[i]);
            j++;
          }
          else if (Sage.DBG) System.out.println("addSeriesInfo: Attempting to add null person or character!  Series=" + series.getTitle() + ", person #" + i);
        }
      }

New code at: https://www.dropbox.com/s/g3p5k0n16v...ix_100918?dl=0

Last edited by wnjj; 10-09-2018 at 09:48 AM.
Reply With Quote
  #33  
Old 10-09-2018, 11:15 AM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,112
The Dragon Prince - Import Video (Netflix show)
So I would imagine the info came from BMT

Applied the latest (Sage.jar_9.1.9_seriesinfo_nullperson_fix_100918)

We'll see how it goes.
Reply With Quote
  #34  
Old 10-09-2018, 11:23 AM
wnjj wnjj is online now
Sage Expert
 
Join Date: Jan 2009
Posts: 691
Quote:
Originally Posted by graywolf View Post
The Dragon Prince - Import Video (Netflix show)
So I would imagine the info came from BMT

Applied the latest (Sage.jar_9.1.9_seriesinfo_nullperson_fix_100918)

We'll see how it goes.
Looks like a there's a bug in BMT then that created these NULL people.
Reply With Quote
  #35  
Old 10-15-2018, 07:20 PM
RymCo RymCo is offline
Sage User
 
Join Date: Jul 2009
Location: Langton, Ontario
Posts: 10
I'm still getting EPG errors trying to load guide data:
Code:
Mon 10/15 21:09:58.184 [EPG@910c30] Saving properties file to C:\Program Files (x86)\SageTV\SageTV\Sage.properties
Mon 10/15 21:09:58.247 [EPG@910c30] Done writing out the data to the properties file
Mon 10/15 21:09:58.257 [EPG@910c30] sage.epg.sd.SDRipper@d61dc3 needs an update in 0:00:00
Mon 10/15 21:09:58.268 [EPG@910c30] EPG needs an update in 0 minutes
Mon 10/15 21:09:58.279 [EPG@910c30] EPG attempting to expand Shaw Direct (Classic) - Satellite - Canada (East) (sdepg)
Mon 10/15 21:09:58.289 [EPG@910c30] expand called on Shaw Direct (Classic) - Satellite - Canada (East) (sdepg) at Mon 10/15 21:09:58.289 expandedUntil=Mon 10/15 21:05:58.113 scannedUntil=Sun 8/17 2:12:55.807
Mon 10/15 21:09:58.300 [EPG@910c30] EPG Expanding Shaw Direct (Classic) - Satellite - Canada (East) (sdepg) at Mon 10/15 21:09:58.300
Mon 10/15 21:11:58.332 [EPG@910c30] SDEPG Exception thrown: Read timed out
Mon 10/15 21:11:58.344 [EPG@910c30] java.net.SocketTimeoutException: Read timed out
Mon 10/15 21:11:58.345 [EPG@910c30] 	at java.net.SocketInputStream.socketRead0(Native Method)
Mon 10/15 21:11:58.345 [EPG@910c30] 	at java.net.SocketInputStream.read(Unknown Source)
Mon 10/15 21:11:58.345 [EPG@910c30] 	at java.net.SocketInputStream.read(Unknown Source)
Mon 10/15 21:11:58.345 [EPG@910c30] 	at sun.security.ssl.InputRecord.readFully(Unknown Source)
Mon 10/15 21:11:58.345 [EPG@910c30] 	at sun.security.ssl.InputRecord.read(Unknown Source)
Mon 10/15 21:11:58.345 [EPG@910c30] 	at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
Mon 10/15 21:11:58.345 [EPG@910c30] 	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
Mon 10/15 21:11:58.345 [EPG@910c30] 	at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
Mon 10/15 21:11:58.345 [EPG@910c30] 	at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
Mon 10/15 21:11:58.346 [EPG@910c30] 	at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
Mon 10/15 21:11:58.346 [EPG@910c30] 	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
Mon 10/15 21:11:58.346 [EPG@910c30] 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
Mon 10/15 21:11:58.346 [EPG@910c30] 	at java.net.HttpURLConnection.getResponseCode(Unknown Source)
Mon 10/15 21:11:58.346 [EPG@910c30] 	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
Mon 10/15 21:11:58.346 [EPG@910c30] 	at sage.epg.sd.SDSageSession.get(SDSageSession.java:196)
Mon 10/15 21:11:58.346 [EPG@910c30] 	at sage.epg.sd.SDSageSession.get(SDSageSession.java:172)
Mon 10/15 21:11:58.346 [EPG@910c30] 	at sage.epg.sd.SDSession.getAuth(SDSession.java:483)
Mon 10/15 21:11:58.346 [EPG@910c30] 	at sage.epg.sd.SDSession.getAuthJson(SDSession.java:690)
Mon 10/15 21:11:58.346 [EPG@910c30] 	at sage.epg.sd.SDSession.getAccountLineups(SDSession.java:812)
Mon 10/15 21:11:58.346 [EPG@910c30] 	at sage.epg.sd.SDRipper.extractGuide(SDRipper.java:1195)
Mon 10/15 21:11:58.347 [EPG@910c30] 	at sage.EPGDataSource.expand(EPGDataSource.java:709)
Mon 10/15 21:11:58.347 [EPG@910c30] 	at sage.EPG.run(EPG.java:773)
Mon 10/15 21:11:58.347 [EPG@910c30] 	at java.lang.Thread.run(Unknown Source)
Mon 10/15 21:11:58.347 [EPG@910c30] ERROR Updating EPG Source Shaw Direct (Classic) - Satellite - Canada (East) (sdepg)
Mon 10/15 21:11:58.357 [EPG@910c30] ErrorText:expand called on Shaw Direct (Classic) - Satellite - Canada (East) (sdepg) at Mon 10/15 21:09:58.289 expandedUntil=Mon 10/15 21:05:58.113 scannedUntil=Sun 8/17 2:12:55.807
EPG Expanding Shaw Direct (Classic) - Satellite - Canada (East) (sdepg) at Mon 10/15 21:09:58.311
My SageTV server is running Java 7 Update 79. Should I upgrade that? Not sure what's going on... I've updated to SageTV 9.1.10. Cleared out my Wiz.bin (backup up first) too.
Reply With Quote
  #36  
Old 10-28-2018, 09:05 PM
wnjj wnjj is online now
Sage Expert
 
Join Date: Jan 2009
Posts: 691
Quote:
Originally Posted by graywolf View Post
The Dragon Prince - Import Video (Netflix show)
So I would imagine the info came from BMT

Applied the latest (Sage.jar_9.1.9_seriesinfo_nullperson_fix_100918)

We'll see how it goes.
Has it been crash-free with the latest code?
Reply With Quote
  #37  
Old 10-29-2018, 05:32 PM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,112
Correct. No crashes since.
Reply With Quote
  #38  
Old 11-05-2018, 03:59 PM
wnjj wnjj is online now
Sage Expert
 
Join Date: Jan 2009
Posts: 691
Quote:
Originally Posted by graywolf View Post
Correct. No crashes since.
The fix for this has been pulled into the main codebase if you want to close the issue on Github.
Reply With Quote
  #39  
Old 11-09-2018, 03:10 PM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,112
Quote:
Originally Posted by wnjj View Post
The fix for this has been pulled into the main codebase if you want to close the issue on Github.
Thanks.
Closed the Issue on Github
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Web Server recordings - NullPointerException pjwerdna SageTV v9 Customizations 1 01-15-2017 05:39 AM
NullPointerException on arrow up/down michaeldjcox SageTV Linux 1 08-18-2015 01:07 AM
Sage Client has NullPointerException and won't connect srothwell SageTV Software 5 04-21-2009 10:10 PM
NullPointerException in SageMC 6.3.9 Julianus SageMC Custom Interface 3 04-10-2009 05:15 PM
NullPointerException importing dvd profiler info skiingwiz SageMC Custom Interface 1 08-27-2008 07:01 PM


All times are GMT -6. The time now is 10:14 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.