Schedules Direct update interval increased to 24 hours

[BACON]

I have been using Schedules Direct since the summer of 2017 without any issues. The last six weeks or so, however, whenever I look at the System Information screen I've noticed that the time referenced by the Next EPG Update In: value always adds up to 24 hours after the Last EPG Update: value, whereas it seemed to be something on the order of every 4 hours in the past. The program guide is still being updated correctly and extends for the same period of time (~17 days), it's just the update interval is much longer now.

The only changes I've made are last month I did a couple Full Channel Scans on my lone lineup due to over-the-air channels in my area moving to new frequencies. I took a look at my Sage.properties and found these values...

epg_data_scan_period=14400000
sdepg_core/editorial/update_interval=86400000

...which I see correspond to 4 hours and 24 hours, respectively, in milliseconds. I looked at some backups, though, and found that's what those values have always been.

I looked through my log files and didn't find anything unexpected during the EPG update section. Is there some other setting that controls these update intervals? Is that interval determined solely by the client, or does the Schedules Direct server specify when the client should check back again?

[graywolf]

Always thought it was 24 hours after last competition

[SHS]

Originally Posted by BACON

I have been using Schedules Direct since the summer of 2017 without any issues. The last six weeks or so, however, whenever I look at the System Information screen I've noticed that the time referenced by the Next EPG Update In: value always adds up to 24 hours after the Last EPG Update: value, whereas it seemed to be something on the order of every 4 hours in the past. The program guide is still being updated correctly and extends for the same period of time (~17 days), it's just the update interval is much longer now.

The only changes I've made are last month I did a couple Full Channel Scans on my lone lineup due to over-the-air channels in my area moving to new frequencies. I took a look at my Sage.properties and found these values...

epg_data_scan_period=14400000
sdepg_core/editorial/update_interval=86400000

...which I see correspond to 4 hours and 24 hours, respectively, in milliseconds. I looked at some backups, though, and found that's what those values have always been.

I looked through my log files and didn't find anything unexpected during the EPG update section. Is there some other setting that controls these update intervals? Is that interval determined solely by the client, or does the Schedules Direct server specify when the client should check back again?

The epg_data_scan_period=14400000 dose correspond to 4 hours but that is for OTA scan EPG data unless epg_name correspond to some thing else then it over ride by SD EPG data or any other EPG plugin.
It possible that the channel is not on SD line so relay on OTA data

If check under setup in your system information page by scroll down to Next EPG Update in: ?????? it should rough idea
Other then that I wouldn't worry about

[BACON]

Originally Posted by SHS

If check under setup in your system information page by scroll down to Next EPG Update in: ?????? it should rough idea
Other then that I wouldn't worry about

That's exactly the problem, though: when I check the Next EPG Update in: value it used to always add up to 4 hours after the Last EPG Update: value whereas lately it's been indicating 24 hour update intervals. I'm curious why that behavior has changed all of a sudden, but I'd also like to get it back to shorter intervals (if they are supposed to be and permitted to be shorter) because sometimes programs can change less than a day in advance.

I found a thread from when Schedules Direct support was first added to SageTV and this post makes the case for the interval being as short as it is, although that's in reference to updates (only) once per day. Is the issue really that, whatever the reason was for my system to download EPG updates every 4 hours, they were actually too frequent and somehow it's now "fixed" itself?

[UgaData]

I believe by default, if you use an EPG provider (Schedules Direct or the now defunct service provided by SageTV) the update interval is 24 hrs.

For some, it is possible to get EPG data from the broadcast stream using an antenna. The downside is there isn't as much look ahead data available. The Over-The-Air interval maybe 4 hrs.

[SHS]

Originally Posted by UgaData

I believe by default, if you use an EPG provider (Schedules Direct or the now defunct service provided by SageTV) the update interval is 24 hrs.

For some, it is possible to get EPG data from the broadcast stream using an antenna. The downside is there isn't as much look ahead data available. The Over-The-Air interval maybe 4 hrs.

You kind need that with some channel as they can be as little as 6 hour to much 48 hour depend on the tv station but sagetv getting really about not grabbing all the data cross all channel as good as NextPVR or SichboPVR even WinTV as start adding EPG in there new ver of WinTV 10 but at leases works more less.

[graywolf]

You could setup a scheduled task to run every # hours that forces an epg update

[BACON]

So could the Last EPG Update:/Next EPG Update in: values refer to either OTA- or internet-based EPG updates?

Since upgrading to Windows 10 a few years ago I'd get numerous "The capture device input Hauppauge WinTV-7164 Analog Capture Digital TV Tuner had a failure while trying to start scanning for data on its channels." warning messages each day, as described here. Thinking that some day I'd get around to debugging/troubleshooting the supposed driver issue, I never applied the disable_data_scanning=true fix, so it's still been scanning (or attempting to) and causing warnings all these years.

Come to think of it, I haven't been seeing those warning messages lately, either. Is it possible that all this time the updates every 4 hours (as reported by the System Information screen) referred to the OTA updates? And, now, after the channel frequency reallocation either that channel no longer carries EPG data or otherwise isn't causing issues with my hardware, so the only updates left for the UI to report are those every 24 hours from Schedules Direct, just as they always were?

[SHS]

Originally Posted by BACON

So could the Last EPG Update:/Next EPG Update in: values refer to either OTA- or internet-based EPG updates?

Since upgrading to Windows 10 a few years ago I'd get numerous "The capture device input Hauppauge WinTV-7164 Analog Capture Digital TV Tuner had a failure while trying to start scanning for data on its channels." warning messages each day, as described here. Thinking that some day I'd get around to debugging/troubleshooting the supposed driver issue, I never applied the disable_data_scanning=true fix, so it's still been scanning (or attempting to) and causing warnings all these years.

Come to think of it, I haven't been seeing those warning messages lately, either. Is it possible that all this time the updates every 4 hours (as reported by the System Information screen) referred to the OTA updates? And, now, after the channel frequency reallocation either that channel no longer carries EPG data or otherwise isn't causing issues with my hardware, so the only updates left for the UI to report are those every 24 hours from Schedules Direct, just as they always were?

Yup that what I think what has happen and like said don't worry about as long sagetv is doing it job as you want then your good to go.

[graywolf]

if you did want it more frequent than 24 hours for SchedulesDirect EPG, you could setup a bat file (if windows) and schedule it when you want.
I do this (once a day) so that it always kicks off at the same time daily instead of 24 hours after the last time it completed (start time then creeps).

C:\GnuWin32\bin\wget.exe "http://userid:password@sage-server:port#/sage/GlobalCommand?command=ForceEpgUpdate"

just replace with your sage webui info for userid, password, sage-server or IP, and the port#

[KryptoNyte]

Just out of curiosity, why do folks want EPG updates more than 24 hours at a time? I don't believe I've ever had a scheduled recording change within 24 hours of the recording time in all the years I've been using Sage.

I have way more problems with the SD EPG data reporting shows as first runs even when they aren't first runs. I never noticed how much of a problem this really is until I started using Channels DVR EPG.

[SHS]

Originally Posted by KryptoNyte

Just out of curiosity, why do folks want EPG updates more than 24 hours at a time? I don't believe I've ever had a scheduled recording change within 24 hours of the recording time in all the years I've been using Sage.

I have way more problems with the SD EPG data reporting shows as first runs even when they aren't first runs. I never noticed how much of a problem this really is until I started using Channels DVR EPG.

Going depend on who EPG it is
The problem doesn't come from SD the problem come from the outside

[Tiki]

Originally Posted by KryptoNyte

Just out of curiosity, why do folks want EPG updates more than 24 hours at a time? I don't believe I've ever had a scheduled recording change within 24 hours of the recording time in all the years I've been using Sage.

I have way more problems with the SD EPG data reporting shows as first runs even when they aren't first runs. I never noticed how much of a problem this really is until I started using Channels DVR EPG.

I’ve actually had quite a few times where EPG data changed after a recording was made. Usually this has been due to last minute schedule changes such as a Presidential address. In most of these instances the schedule change should have been known hours or even days in advance, but Sage didn’t see the change until it was too late. It’s possible that more frequent refreshes would help in some of these cases.

[Carlton Bale]

This issue caused me to miss recording the first half of Game 7 of the World Series. The EPG data initially assumed no Game 7, and the EPG didn't update so reflect the schedule change. I ran a manual refresh, the EPG data updated, and Sage started recording mid-game.

Should this just be a variable that could be updated in the source code? It theoretically seems like something that should be easy for someone with basic knowledge of the code to update.

I'm glad there's a workaround to schedule a batch refresh via wget to the web server, but it's not the ideal solution.

[wnjj]

Originally Posted by Carlton Bale

This issue caused me to miss recording the first half of Game 7 of the World Series. The EPG data initially assumed no Game 7, and the EPG didn't update so reflect the schedule change. I ran a manual refresh, the EPG data updated, and Sage started recording mid-game.

Should this just be a variable that could be updated in the source code? It theoretically seems like something that should be easy for someone with basic knowledge of the code to update.

I'm glad there's a workaround to schedule a batch refresh via wget to the web server, but it's not the ideal solution.

I’m pretty sure it’s hardcoded to 24 hours right here: https://github.com/google/sagetv/blo...e/EPG.java#L50

I don’t know if SD would be too thrilled if a bunch of users started updating too frequently. It’s too bad there isn’t a SD push-based incremental update for these last minute changes.

[graywolf]

Originally Posted by KryptoNyte

Just out of curiosity, why do folks want EPG updates more than 24 hours at a time? I don't believe I've ever had a scheduled recording change within 24 hours of the recording time in all the years I've been using Sage.

I only do it once per day but didn't like the start time creep. That is the main reason I use the scheduled task/bat file. This way it consistently starts same time every day.

[BACON]

Originally Posted by KryptoNyte

Just out of curiosity, why do folks want EPG updates more than 24 hours at a time? I don't believe I've ever had a scheduled recording change within 24 hours of the recording time in all the years I've been using Sage.

The "if necessary" games at the end of a playoff series, if they end up not being necessary, would be one example of programming that can change less than 24 hours in advance. Programs that get preempted and (re-)air in the middle of the night would be another. Local programming seems most likely to fall victim to last-minute scheduling shenanigans. Whether those changes actually get communicated by broadcasters to wherever Schedules Direct sources its data and in time, however, is another matter...

Originally Posted by wnjj

I’m pretty sure it’s hardcoded to 24 hours right here: https://github.com/google/sagetv/blo...e/EPG.java#L50

I don’t know if SD would be too thrilled if a bunch of users started updating too frequently. It’s too bad there isn’t a SD push-based incremental update for these last minute changes.

According to the Schedules Direct FAQ (first question) and this post on their forums a block will result after 200 requests in a 2 hour period, though it notes one logical request operation may in result 10 or more requests. With the limit that high it seems that updating a mere 2, even 3 times per day wouldn't hurt anything, though it also says "Data is updated once a day" so I guess it depends on when a device's daily request happens in relation to Schedule Direct's data update (e.g. a client updates once per day at 6 AM and new data becomes available at 9 AM).

I would be tempted to try changing to 12-hour intervals, but evidently sdepg_core/editorial/update_interval= doesn't control that like I initially thought. I would also wonder about then causing the client to reprocess the (entire?) schedule more often. With a schedule reaching out 17 days I don't really care about getting the next 12-hour interval at the end of the 17 days 12 hours sooner; I'm more interested in picking up any changes in the 12- or 24-hour period starting now. It would be nice if there were two schedules - one for updates that extend the program guide period, and one for updates that only fetch changes, if any, to the period it already knows about (or a subset of it, like only the next 12 or 24 hours) - but then I don't enough about the inner workings of SageTV's scheduler or the Schedules Direct API to know if that's feasible.

[Carlton Bale]

Originally Posted by BACON

According to the Schedules Direct FAQ (first question) and this post on their forums a block will result after 200 requests in a 2 hour period, though it notes one logical request operation may in result 10 or more requests. With the limit that high it seems that updating a mere 2, even 3 times per day wouldn't hurt anything

I agree that updating 3 times per day shouldn't be an issue with Schedules Direct. It should also solve the problem of missed schedule changes.

Anyone able to do a source code commit?

[Malfunction]

Originally Posted by graywolf

if you did want it more frequent than 24 hours for SchedulesDirect EPG, you could setup a bat file (if windows) and schedule it when you want.
I do this (once a day) so that it always kicks off at the same time daily instead of 24 hours after the last time it completed (start time then creeps).

C:\GnuWin32\bin\wget.exe "http://userid:password@sage-server:port#/sage/GlobalCommand?command=ForceEpgUpdate"

just replace with your sage webui info for userid, password, sage-server or IP, and the port#

How do I know what port # to use? I got 64 bit wget.exe, got uid, password, etc. I've been trying to make SageTV update the epg. Mine only updates when I start SageTV.

[graywolf]

Use the port you have for the Sage WebUI