SageTV Community  

Go Back   SageTV Community > SageTV Products > SageTV EPG Service

Notices

SageTV EPG Service Discussion related to the SageTV EPG Service used within SageTV. Questions about service area coverage, channel lineups, EPG listings, XMLTV, or anything else related to the service or programming guide data for SageTV should be posted here.

Reply
 
Thread Tools Search this Thread Display Modes
  #321  
Old 06-17-2017, 11:45 AM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,366
Quote:
Originally Posted by ranger View Post
Perhaps. But it does this EVERY time I look at the page. I can come back an hour later and it will again show the current time and 0 minutes to next update. Either it is continually updating the EPG (24x7), or something is telling this screen (and the web interface) that the EPG is updating. A problem no matter which way you look at it.

SageTV version 9.1.2.662 is running on both my servers and only one of them behaves in this manner.
I have this same on one of my Sage boxes but not another.
Both SD EPG.

Turned on JSON logging for sdepg and I see the following in the sd_epg.log on the pc having issues

Code:
#### Exception Start ####-1
java.lang.ArrayIndexOutOfBoundsException: -1
	at sage.epg.sd.SDRecommended.getRecommendations(SDRecommended.java:365)
	at sage.epg.sd.SDRipper.extractGuide(SDRipper.java:2375)
	at sage.EPGDataSource.expand(EPGDataSource.java:709)
	at sage.EPG.run(EPG.java:783)
	at java.lang.Thread.run(Unknown Source)

#### Exception End ####
Reply With Quote
  #322  
Old 06-17-2017, 12:14 PM
Tiki's Avatar
Tiki Tiki is offline
Sage Icon
 
Join Date: Feb 2005
Location: Southwest Florida, USA
Posts: 1,991
Quote:
Originally Posted by ranger View Post
Perhaps. But it does this EVERY time I look at the page. I can come back an hour later and it will again show the current time and 0 minutes to next update. Either it is continually updating the EPG (24x7), or something is telling this screen (and the web interface) that the EPG is updating. A problem no matter which way you look at it.

SageTV version 9.1.2.662 is running on both my servers and only one of them behaves in this manner.
It normally takes more than an hour to complete the update.
__________________
Server: Ryzen 2400G with integrated graphics, ASRock X470 Taichi Motherboard, HDMI output to Vizio 1080p LCD, Win10-64Bit (Professional), 16GB RAM
Capture Devices (7 tuners): Colossus (x1), HDHR Prime (x2)
,USBUIRT (multi-zone)
Source:
Comcast/Xfinity X1 Cable
Primary Client: Server Other Clients: (1) HD200, (1) HD300
Retired Equipment: MediaMVP, PVR150 (x2), PVR150MCE,
HDHR, HVR-2250, HD-PVR
Reply With Quote
  #323  
Old 06-17-2017, 12:16 PM
ranger ranger is offline
Sage Aficionado
 
Join Date: Nov 2003
Location: Virginia
Posts: 256
Quote:
Originally Posted by Tiki View Post
It normally takes more than an hour to complete the update.
Thanks, but it doesn't matter WHEN I look at this. It always says the same thing. Two, three, four hours later. No difference.
Reply With Quote
  #324  
Old 06-17-2017, 12:27 PM
wnjj wnjj is offline
Sage Icon
 
Join Date: Jan 2009
Posts: 1,475
Quote:
Originally Posted by graywolf View Post
I have this same on one of my Sage boxes but not another.
Both SD EPG.

Turned on JSON logging for sdepg and I see the following in the sd_epg.log on the pc having issues

Code:
#### Exception Start ####-1
java.lang.ArrayIndexOutOfBoundsException: -1
	at sage.epg.sd.SDRecommended.getRecommendations(SDRecommended.java:365)
	at sage.epg.sd.SDRipper.extractGuide(SDRipper.java:2375)
	at sage.EPGDataSource.expand(EPGDataSource.java:709)
	at sage.EPG.run(EPG.java:783)
	at java.lang.Thread.run(Unknown Source)

#### Exception End ####
Looks like that bug was fixed 7 days ago. Should be fixed in 9.1.4.

https://github.com/google/sagetv/com...c8aa9a681df5b0

Last edited by wnjj; 06-17-2017 at 12:30 PM.
Reply With Quote
  #325  
Old 06-17-2017, 12:44 PM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,366
Quote:
Originally Posted by wnjj View Post
Looks like that bug was fixed 7 days ago. Should be fixed in 9.1.4.

https://github.com/google/sagetv/com...c8aa9a681df5b0
Thanks
I'll keep an eye out on the Windows Installer thread for when 9.1.4 is available
Reply With Quote
  #326  
Old 06-17-2017, 12:59 PM
EnterNoEscape's Avatar
EnterNoEscape EnterNoEscape is offline
SageTVaholic
 
Join Date: Jun 2010
Location: Harrisburg, PA
Posts: 2,656
Quote:
Originally Posted by graywolf View Post
I have this same on one of my Sage boxes but not another.
Both SD EPG.

Turned on JSON logging for sdepg and I see the following in the sd_epg.log on the pc having issues

Code:
#### Exception Start ####-1
java.lang.ArrayIndexOutOfBoundsException: -1
	at sage.epg.sd.SDRecommended.getRecommendations(SDRecommended.java:365)
	at sage.epg.sd.SDRipper.extractGuide(SDRipper.java:2375)
	at sage.EPGDataSource.expand(EPGDataSource.java:709)
	at sage.EPG.run(EPG.java:783)
	at java.lang.Thread.run(Unknown Source)

#### Exception End ####
That situation is cleaned up in version 9.1.4.
__________________
SageTV v9 Server: ASRock Z97 Extreme4, Intel i7-4790K @ 4.4Ghz, 32GB RAM, 6x 3TB 7200rpm HD, 2x 5TB 7200rpm HD, 2x 6TB 7200rpm HD, 4x 256GB SSD, 4x 500GB SSD, unRAID Pro 6.7.2 (Dual Parity + SSD Cache).
Capture: 1x Ceton InfiniTV 4 (ClearQAM), 2x Ceton InfiniTV 6, 1x BM1000-HDMI, 1x BM3500-HDMI.

Clients: 1x HD300 (Living Room), 1x HD200 (Master Bedroom).
Software: OpenDCT :: WMC Live TV Tuner :: Schedules Direct EPG
Reply With Quote
  #327  
Old 06-17-2017, 01:18 PM
EnterNoEscape's Avatar
EnterNoEscape EnterNoEscape is offline
SageTVaholic
 
Join Date: Jun 2010
Location: Harrisburg, PA
Posts: 2,656
On the same note about exceptions in EPG data processing, I think SageTV should tone down the retry times by default. I think at the moment, it retries every few minutes. Something like once an hour would make the most sense to me. That might be something to be revisited after 7/1.
__________________
SageTV v9 Server: ASRock Z97 Extreme4, Intel i7-4790K @ 4.4Ghz, 32GB RAM, 6x 3TB 7200rpm HD, 2x 5TB 7200rpm HD, 2x 6TB 7200rpm HD, 4x 256GB SSD, 4x 500GB SSD, unRAID Pro 6.7.2 (Dual Parity + SSD Cache).
Capture: 1x Ceton InfiniTV 4 (ClearQAM), 2x Ceton InfiniTV 6, 1x BM1000-HDMI, 1x BM3500-HDMI.

Clients: 1x HD300 (Living Room), 1x HD200 (Master Bedroom).
Software: OpenDCT :: WMC Live TV Tuner :: Schedules Direct EPG
Reply With Quote
  #328  
Old 06-17-2017, 03:27 PM
wnjj wnjj is offline
Sage Icon
 
Join Date: Jan 2009
Posts: 1,475
Quote:
Originally Posted by EnterNoEscape View Post
On the same note about exceptions in EPG data processing, I think SageTV should tone down the retry times by default. I think at the moment, it retries every few minutes. Something like once an hour would make the most sense to me. That might be something to be revisited after 7/1.
Agree. While we're at it, why not make a 'manual refresh' option like we have for imported media? While not often, there's just enough times were a refresh is the right answer. Turning off and on a channel in the lineup isn't intuitive for new users.
Reply With Quote
  #329  
Old 06-17-2017, 03:36 PM
Taddeusz Taddeusz is offline
SageTVaholic
 
Join Date: Nov 2004
Location: Yukon, OK
Posts: 3,919
Quote:
Originally Posted by wnjj View Post
Agree. While we're at it, why not make a 'manual refresh' option like we have for imported media? While not often, there's just enough times were a refresh is the right answer. Turning off and on a channel in the lineup isn't intuitive for new users.
There is already an option in the web interface to force an EPG update. I'll second that there should be one in the STV. Removing and adding back a channel isn't exactly an intuitive way to force an update.
__________________
Server: i5 8400, ASUS Prime H370M-Plus/CSM, 16GB RAM, 15TB drive array + 500GB cache, 2 HDHR's, SageTV 9, unRAID 6.6.3
Client 1: HD300 (latest FW), HDMI to an Insignia 65" 1080p LCD and optical SPDIF to a Sony Receiver
Client 2: HD200 (latest FW), HDMI to an Insignia NS-LCD42HD-09 1080p LCD
Reply With Quote
  #330  
Old 06-18-2017, 08:48 AM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,366
Code:
[0002209] [SageTV log     ] Sun 6/18 10:45:12.855 [pool-4-thread-19@abbedd] SDEPG Unable to get head-shots for: SDPerson{personId='369894', nameId='378769', name='Michael S. Glick', role='Executive Producer', characterName='null', billingOrder='03'}
[0002210] [SageTV log     ] Sun 6/18 10:45:12.862 [pool-4-thread-19@abbedd] java.net.ConnectException: Connection timed out: connect
Is that normal/expected?
If so, might want to add it to the list of "messages" to suppress from the SageTV logs.
Reply With Quote
  #331  
Old 06-18-2017, 05:48 PM
EnterNoEscape's Avatar
EnterNoEscape EnterNoEscape is offline
SageTVaholic
 
Join Date: Jun 2010
Location: Harrisburg, PA
Posts: 2,656
Quote:
Originally Posted by graywolf View Post
Code:
[0002209] [SageTV log     ] Sun 6/18 10:45:12.855 [pool-4-thread-19@abbedd] SDEPG Unable to get head-shots for: SDPerson{personId='369894', nameId='378769', name='Michael S. Glick', role='Executive Producer', characterName='null', billingOrder='03'}
[0002210] [SageTV log     ] Sun 6/18 10:45:12.862 [pool-4-thread-19@abbedd] java.net.ConnectException: Connection timed out: connect
Is that normal/expected?
If so, might want to add it to the list of "messages" to suppress from the SageTV logs.
Yes, this is normal and will not cause EPG retrieval to be considered a failure. I don't believe in suppressing most exception messages if you're not going to handle them in any special way when the exception happens because then you're creating a gap in logging that might be significant in future diagnosis of a problem. In this case each entry you see is for a different person, so it's not really possible to reduce this down to something like a note of how many times it happened.

If you're seeing enough of these that your logs are actually rolling over from them alone, that might be an indication that we should adjust the thread count for everyone. I did tune this feature on a 75/75 FiOS connection, so I might have over-estimated what most setups will be able to handle. Either way, you can see how this has already lead to a discussion we wouldn't even be having without the exceptions being logged. Alternatively we'd be having a discussion where someone is asking why does this person have a headshot sometimes and other times no headshot and we wouldn't have anything in the logs to help us.

Basically there was a whole discussion about getting these headshots on GitHub whereby I reported that checking all possible headshots takes hours. We decided on multi-threading it since it's more of a latency problem because each check is it's own connection/handshake. That drove the time spent down a lot.

I'll just namedrop here. Narflex asked if we could have headshots since SageTV actually already had the support for it in the STV. He helped me work out some of the details about the Person objects and requested that we not perform lookups for celebrity images on demand and that instead the Person objects already have the URL that will take us directly to the image.

Schedules Direct doesn't provide any way to know if a celebrity image exists without just checking and they don't provide a way to batch check if the images will exist. We have to check each one individually and that's always thousands. This blast of checking happens after all important guide data has been imported and is limited to people that were in the shows we just update/added. It is required because otherwise I would not be able to meet the requirement to have the image URL already resolved.
__________________
SageTV v9 Server: ASRock Z97 Extreme4, Intel i7-4790K @ 4.4Ghz, 32GB RAM, 6x 3TB 7200rpm HD, 2x 5TB 7200rpm HD, 2x 6TB 7200rpm HD, 4x 256GB SSD, 4x 500GB SSD, unRAID Pro 6.7.2 (Dual Parity + SSD Cache).
Capture: 1x Ceton InfiniTV 4 (ClearQAM), 2x Ceton InfiniTV 6, 1x BM1000-HDMI, 1x BM3500-HDMI.

Clients: 1x HD300 (Living Room), 1x HD200 (Master Bedroom).
Software: OpenDCT :: WMC Live TV Tuner :: Schedules Direct EPG
Reply With Quote
  #332  
Old 06-18-2017, 07:36 PM
jpwegas jpwegas is offline
Sage Aficionado
 
Join Date: May 2007
Posts: 492
Switched one of my 4 tuners to use SD. After updating, while scrolling through the "Parallel Recording Schedule", the Sage process locked up. Twice. Had to kill -9 to force stop and then restarted the process. Attached the portion of the sagetv_0.txt when it happened.

Maybe I was too quick and the guide isn't finished updating and it's referencing something that hasn't loaded from the SD site yet?

Let me know what other info I can provide to help debug.

--John
Attached Files
File Type: txt sage.hang.2017-06-18.txt (45.0 KB, 118 views)
Reply With Quote
  #333  
Old 06-18-2017, 07:56 PM
EnterNoEscape's Avatar
EnterNoEscape EnterNoEscape is offline
SageTVaholic
 
Join Date: Jun 2010
Location: Harrisburg, PA
Posts: 2,656
Quote:
Originally Posted by jpwegas View Post
Switched one of my 4 tuners to use SD. After updating, while scrolling through the "Parallel Recording Schedule", the Sage process locked up. Twice. Had to kill -9 to force stop and then restarted the process. Attached the portion of the sagetv_0.txt when it happened.

Maybe I was too quick and the guide isn't finished updating and it's referencing something that hasn't loaded from the SD site yet?

Let me know what other info I can provide to help debug.

--John
The logs do not appear to indicate this is a Schedules Direct issue. It appears to be an issue loading an image.
__________________
SageTV v9 Server: ASRock Z97 Extreme4, Intel i7-4790K @ 4.4Ghz, 32GB RAM, 6x 3TB 7200rpm HD, 2x 5TB 7200rpm HD, 2x 6TB 7200rpm HD, 4x 256GB SSD, 4x 500GB SSD, unRAID Pro 6.7.2 (Dual Parity + SSD Cache).
Capture: 1x Ceton InfiniTV 4 (ClearQAM), 2x Ceton InfiniTV 6, 1x BM1000-HDMI, 1x BM3500-HDMI.

Clients: 1x HD300 (Living Room), 1x HD200 (Master Bedroom).
Software: OpenDCT :: WMC Live TV Tuner :: Schedules Direct EPG
Reply With Quote
  #334  
Old 06-18-2017, 08:14 PM
jpwegas jpwegas is offline
Sage Aficionado
 
Join Date: May 2007
Posts: 492
Quote:
Originally Posted by EnterNoEscape View Post
The logs do not appear to indicate this is a Schedules Direct issue. It appears to be an issue loading an image.
Agreed, it does appear to be due to an image load issue, but it seems related to setting up the new lineup, even if the SD plugin wasn't the direct cause.

Perhaps something in the new lineup referenced an image that hadn't been downloaded yet?

I haven't had a hard hang like that, especially twice in 10 minutes, in years. It happened right as I scrolled into view the only upcoming show on the tuner using the new lineup.

I have a core dump too I can send if someone would be able to analyze it.

--John
Reply With Quote
  #335  
Old 06-18-2017, 08:26 PM
jpwegas jpwegas is offline
Sage Aficionado
 
Join Date: May 2007
Posts: 492
Quote:
Originally Posted by jpwegas View Post
I have a core dump too I can send if someone would be able to analyze it.
Looking closer, the core dump was truncated (possibly due to me killing the process) so I'm not sure it's useful.
Reply With Quote
  #336  
Old 06-18-2017, 09:15 PM
EnterNoEscape's Avatar
EnterNoEscape EnterNoEscape is offline
SageTVaholic
 
Join Date: Jun 2010
Location: Harrisburg, PA
Posts: 2,656
Quote:
Originally Posted by jpwegas View Post
Agreed, it does appear to be due to an image load issue, but it seems related to setting up the new lineup, even if the SD plugin wasn't the direct cause.

Perhaps something in the new lineup referenced an image that hadn't been downloaded yet?

I haven't had a hard hang like that, especially twice in 10 minutes, in years. It happened right as I scrolled into view the only upcoming show on the tuner using the new lineup.

I have a core dump too I can send if someone would be able to analyze it.

--John
Knowing a little about how images are loaded, this isn't possible. A show is added with the image URLs. SageTV downloads the image via the URL when the STV wants the image. There isn't a timing issue with the Schedules Direct EPG data. There is a problem more likely with loading the image after it was downloaded. The problem could have been because SageTV can't load the image correctly as it was downloaded.
__________________
SageTV v9 Server: ASRock Z97 Extreme4, Intel i7-4790K @ 4.4Ghz, 32GB RAM, 6x 3TB 7200rpm HD, 2x 5TB 7200rpm HD, 2x 6TB 7200rpm HD, 4x 256GB SSD, 4x 500GB SSD, unRAID Pro 6.7.2 (Dual Parity + SSD Cache).
Capture: 1x Ceton InfiniTV 4 (ClearQAM), 2x Ceton InfiniTV 6, 1x BM1000-HDMI, 1x BM3500-HDMI.

Clients: 1x HD300 (Living Room), 1x HD200 (Master Bedroom).
Software: OpenDCT :: WMC Live TV Tuner :: Schedules Direct EPG
Reply With Quote
  #337  
Old 06-18-2017, 10:06 PM
jpwegas jpwegas is offline
Sage Aficionado
 
Join Date: May 2007
Posts: 492
I avoided the Scheduled Recordings screen for a while and whatever was causing it I can't reproduce it any longer.

Thanks,

--John
Reply With Quote
  #338  
Old 06-19-2017, 05:51 AM
Tiki's Avatar
Tiki Tiki is offline
Sage Icon
 
Join Date: Feb 2005
Location: Southwest Florida, USA
Posts: 1,991
If this was for a recording that was in progress, on some screens Sage will try to create and display a thumbnail from a few second into the actual recording. It uses the Sage Transcoder to generate the thumbnail. I found in the past that some badly formatted Comcast recordings would trip-up the transcoder and cause the Sage UI to hang if it was trying to display that thumbnail. I saw this in the Malore Recording Scedule screen and the Malore current Recordings screen with Sage v7. There may be other screens where this could happen.
__________________
Server: Ryzen 2400G with integrated graphics, ASRock X470 Taichi Motherboard, HDMI output to Vizio 1080p LCD, Win10-64Bit (Professional), 16GB RAM
Capture Devices (7 tuners): Colossus (x1), HDHR Prime (x2)
,USBUIRT (multi-zone)
Source:
Comcast/Xfinity X1 Cable
Primary Client: Server Other Clients: (1) HD200, (1) HD300
Retired Equipment: MediaMVP, PVR150 (x2), PVR150MCE,
HDHR, HVR-2250, HD-PVR
Reply With Quote
  #339  
Old 06-19-2017, 09:32 AM
MickBurke's Avatar
MickBurke MickBurke is offline
Sage Aficionado
 
Join Date: Jan 2005
Posts: 318
So I'm running thought the change for this now, and I'm wondering; is there no way to change the lineup without redoing all the channels?

When i changed the lineup, it resets all the channels back to 'default'.

I already started the one, and I see no 'cancel' so I think I'm stuck with this one anyway, but figured I'd check in...?
Reply With Quote
  #340  
Old 06-20-2017, 07:58 PM
pendragonsound pendragonsound is offline
Sage User
 
Join Date: Sep 2008
Posts: 44
I moved from Sage 7 to 9 a couple of days ago, and partially converted to Schedules Direct today. There's still one batch of channels to convert where no good solution appears to exist, and in some ways I am amplifying MickBurke's post above. I have 4 pairs of tuners with the following sources:

OTA broadcast
Dish Network satellite subscription
Shaw (Canada) satellite subscription
Free-To-Air (FTA) satellites

I converted the first three from line-ups with the old licensed EPG service to the equivalent Schedules Direct line-ups. That was straight-forward, but very tedious. In particular, Dish and Shaw each have close to 1000 physical and virtual channels, and their ranges completely overlap numerically. To solve this in the past, I had remapped all of the Shaw channels so the logical channel numbers are always 1000 larger than the physical channel numbers. Because converting to Schedules Direct requires reimporting the entire channel line-up, I had to manually remap each of the Shaw channels again. Furthermore I don't subscribe to all of the Dish and Shaw channels, so I had to go through each list and manually disable the channels I do not pay for. Fortunately I had saved a channel list from before, and thus I had a guide. But it took me nearly four hours to readjust everything back to the way it was. As MickBurke pointed out, it would have saved a lot of time to keep the channels unaltered when switching over. Anyway that's done and it appears to be working.

Now I am expecting the above problems in spades for the FTA channel conversion. I have dishes pointed at dozens of satellites, with several thousand total channels. Most of these don't connect to an EPG, but there are still a lot that do. There is no 'line-up' that covers any of these channels. With the old EPG service, I found I did not have to specify a line-up to get EPG information for the FTA channels, and thus I did not. Whenever I found the 'Station ID' (5 digit number) for a FTA channel on Zap2It, I simply entered that manually into my FTA channel list. The EPG was populated for these channels even without a formal line-up. I had hoped this might continue working with Schedules Direct, but even though the Station IDs translate over perfectly, it appears to be continuing to pull these through the old, licensed path.

Ideally I would like to leave my FTA channel list alone, but pull the EPG for those channels that can get it from Schedules Direct. I cannot find any way to do this, and would be grateful to anyone who could point me in the right direction. I'm assuming that means my only choice is to (1) import an irrelevant channel line-up for my FTA tuners, (2) disable all of the channels in that line-up, (3) reenter thousands of channels that will never have an EPG, and (4) reenter the Station IDs for the channels where I was able to find them. This will take days, and if MickBurke's concept were to work, it would take minutes.
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Plugin: Schedules Direct EPG (not for v9) Slugger SageTV v7 Customizations 851 07-19-2017 07:48 AM
Schedules Direct Plugin Issue rickgillyon SageTV v9 Customizations 20 09-25-2016 12:03 PM
Schedules Direct EPG Plugin: Call for private beta testers Slugger SageTV v7 Customizations 11 01-19-2013 11:59 AM
Schedules Direct EPG Plugin PoC Completed Slugger SageTV EPG Service 71 12-04-2011 12:12 PM
Schedules Direct or what? Ryel SageTV EPG Service 6 01-25-2008 03:03 PM


All times are GMT -6. The time now is 02:28 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.