Plugin: Schedules Direct EPG (not for v9)

[rkulagow]

It's been solved in the beta API, which I believe Slugger has almost completed support for.

[panteragstk]

Originally Posted by rkulagow

It's been solved in the beta API, which I believe Slugger has almost completed support for.

Thanks

[Slugger]

It has been on the Schedules Direct test server. Those changes will be moved into production this week, which requires me to make updates to the Sage plugin. I've just finished moving, etc., etc. and am hoping to get that plugin update done this week. So I'll say next week for the Sunday Ticket listings, but maybe I should just say it's coming "soon". Honestly, though, it should be next week at some point.

[panteragstk]

Originally Posted by Slugger

It has been on the Schedules Direct test server. Those changes will be moved into production this week, which requires me to make updates to the Sage plugin. I've just finished moving, etc., etc. and am hoping to get that plugin update done this week. So I'll say next week for the Sunday Ticket listings, but maybe I should just say it's coming "soon". Honestly, though, it should be next week at some point.

I really appreciate it. Thanks for working on this.

[Phydeaux]

The plugin has been working flawlessly for me for several months now, however it has suddenly stopped working. No changes to the SageTV server recently. I'm looking at the sdepg.log file and seeing mention of "Account expired". Not sure what this is, as my SD account doesn't expire until March of 2014... Any ideas?

2013-10-10 18:50:56,964 INFO  [ProgramGenerator]: Processed show generators in 33ms
2013-10-10 18:50:57,182 INFO  [EpgDownloader]: [C:\Program Files (x86)\Java\jre7\bin\java, -Xmx512m, -jar, C:\Program Files (x86)\SageTV\SageTV\plugins\sdepg\tools\sdjson.jar, -c, grab, -u, Phydeaux, -p, *****, -o, C:\Program Files (x86)\SageTV\SageTV\plugins\sdepg\sdjson.epg, -a, sagetv-sdepg/0.0.4.2 (Windows 7 x86 6.1), -t, 200, -b, https://data2.schedulesdirect.org, -pc, 50000, -sc, 1000, -l, INFO, -g, C:\Program Files (x86)\SageTV\SageTV\plugins\sdepg\ignore.txt]
2013-10-10 18:52:30,336 ERROR [EpgDownloader]: sdjson download failed! [rc=1]
2013-10-10 18:52:30,337 ERROR [EpgDownloader]: stdout:
INFO - ScheduleTask download DONE! [10624ms]
INFO - ScheduleTask COMPLETE! [14976ms]
INFO - Identified 5652 program ids requiring an update!
ERROR - IOError receiving program data; filling in empty program info!
java.net.UnknownHostException: s3.amazonaws.com
	at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
	at java.net.InetAddress$1.lookupAllHostAddr(Unknown Source)
	at java.net.InetAddress.getAddressesFromNameService(Unknown Source)
	at java.net.InetAddress.getAllByName0(Unknown Source)
	at java.net.InetAddress.getAllByName(Unknown Source)
	at java.net.InetAddress.getAllByName(Unknown Source)
	at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.resolveHostname(DefaultClientConnectionOperator.java:278)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:162)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
	at org.apache.http.client.fluent.Request.execute(Request.java:145)
	at org.schedulesdirect.grabber.ProgramTask.run(ProgramTask.java:82)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

2013-10-10 18:52:30,337 ERROR [EpgDownloader]: stderr:
Exception in thread "main" java.io.IOException: Account expired.
	at org.schedulesdirect.api.NetworkEpgClient.initStatusObjects(NetworkEpgClient.java:204)
	at org.schedulesdirect.api.NetworkEpgClient.getUserStatus(NetworkEpgClient.java:210)
	at org.schedulesdirect.grabber.Grabber.run(Grabber.java:629)
	at org.schedulesdirect.grabber.Grabber.main(Grabber.java:674)

2013-10-10 18:52:30,350 ERROR [EPGImportPluginSchedulesDirect]: Download of EPG data failed!
java.io.IOException:  download failed! [rc=1]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
	at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:71)
	at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callConstructor(ConstructorSite.java:81)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:57)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:182)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:190)
	at sagex.epg.schedulesdirect.io.EpgDownloader.download(EpgDownloader.groovy:83)
	at sagex.epg.schedulesdirect.io.EpgDownloader$download.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
	at sagex.epg.schedulesdirect.EPGImportPluginSchedulesDirect.doUpdate(EPGImportPluginSchedulesDirect.groovy:278)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:272)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:52)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:137)
	at sagex.epg.schedulesdirect.EPGImportPluginSchedulesDirect.updateGuide(EPGImportPluginSchedulesDirect.groovy:164)
	at sage.ae.y(Unknown Source)
	at sage.a2.e(Unknown Source)
	at sage.a9.dK(Unknown Source)
	at sage.ae.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
2013-10-10 18:52:30,385 INFO  [AiringGenerator]: Processed airing generators in 16ms

[Slugger]

The UnknownHostException is a concern. Seems there are DNS/network problems on your end. That's worth investigating.

The account expired message... that's from Schedules Direct so if that continues you will want to open a ticket with them to get that fixed up.

[Phydeaux]

Thanks Slugger. I've now created a ticket at SD, so hopefully they can tell me why it's reporting an expired membership when using JSON. I can log into their website fine and access my lineups there - and my account definitely does not expire until March, 2014 - verified.

This DNS issue though. What the heck? Definitely nothing has changed in my home network for a very long time. I don't seem to have any problem resolving anywhere I need to go from any of the computers here, including the SageTV server. Not sure where to begin looking...

RLW

[Slugger]

If it only appears once in your logs then it's probably just an intermittent issue that can safely be ignored, but if you're getting that UnknownHost error repeatedly, then something's wrong with your inet connection or your DNS server (or your ISP's DNS servers, as the case may be).

[rkulagow]

Originally Posted by Phydeaux

The plugin has been working flawlessly for me for several months now, however it has suddenly stopped working. No changes to the SageTV server recently. I'm looking at the sdepg.log file and seeing mention of "Account expired". Not sure what this is, as my SD account doesn't expire until March of 2014... Any ideas?

I've sent logs to Slugger. It appears that your client is connecting, obtaining an authorization token, requesting work, etc. But then during the session, it's requesting a new token. That invalidates the old token, but then later again the client presents the old token, and that generates the error message.

Are you running something in parallel somehow that's causing new token requests to be sent to the server?

[Phydeaux]

Not that I'm aware of Rob - I've done nothing to the Sage server for several months. It's not used as a workstation or anything, all that's on it is SageTV - otherwise it's a generic Windows 7 Pro machine.

RLW

[Slugger]

If you load the web UI status pages for Schedules Direct while the grabber is running in the background, that will cause this kind of thing (the web UI and the grabber app each get their own tokens, if you run both at the same time the one will invalidate the other's token).

[Phydeaux]

I'll buy that - it's quite probable that I had the web UI status pages minimized on my workstation when I kicked off the EPG update on the SageTV server (by adding a new channel).

But this can't be the reason it stopped working, likely about 2 weeks ago. I had noticed the listings getting closer and closer to running out, but based on previous experience had ignored it thinking that it will refresh before it ran out. But it ran out, hence my attempt to force it to run by stopping SageTV, deleting the epg file and log, restarting SageTV and adding a channel to the lineup to kick it off.

Note that over the last couple weeks as the listing grew shorter, I did check the Last EPG Update date on the UI status page and it was reporting that it was running every day at the same time as always.

So for the sake of argument, let's say the token issue is solved and the DNS resolution was a transient miss.... Any ideas?

RLW

[Slugger]

So you just forced another epg update, it completed, and you still have no data? Now I need to see that new sdepg.log file. Also, is the created sdjson.epg file of suitable size (i.e. on the order of MBs)?

[Phydeaux]

I'll have to look at the file when I get home from work tonight. I'll post again later.

Thanks,
RLW

[Phydeaux]

The sdjson.epg file is 1124kb. I'll re-do what I did the other night, making sure I don't have the web ui open anywhere. Then I'll report back.

RLW

[Phydeaux]

Did the same drill: stopped SageTV, deleted the sdjson.epg file and log files, re-started SageTV then added a channel to kick off the refresh. Worked a charm. So it sounds like the open UI was the cause of the false "expired" warning, due to the token collision. And the not resolving issue was transient.

Leads me to believe that there must have been a corruption in the sdjson.epg file previously. Weird.

Thanks Slugger and Rob!

RLW

[Slugger]

Scheduled for release sometime this week; posting the release notes now before I forget half the stuff.

Support for latest Schedules Direct API version (20130709). New features in this API release from SD:

• S/E metadata returns
• NFL Sunday Ticket listings with specific game data for DirecTV (USA) and Shaw Direct (Canada)
• Channel logos now available (however this sdepg update does not use them; a future fix release of 0.5 will address this)
• Various backend changes to improve performance, etc. on the server side of the service

Changes specific to the plugin:

• Updates to web UI to work with new SD API
• Support changes to sdjson grabber
• Fix 'Refresh EPG' button on plugin config screen (it finally works!)
• Various bug fixes and improvements to filters and generators
• Support for new S/E metadata providers
• Various other bug fixes

The new plugin will be published shortly after SD updates the API on their production servers. After SD updates, you will require 0.5.x of sdepg plugin to communicate with the SD service.

[panteragstk]

Thanks for working on this. It is much appreciated.

[Slugger]

The new version of the plugin will be in the repository shortly (once I click the confirmation link in my inbox). This upgrade will require all users to register their headends with the JSON server once again. This is due to data format changes at SD.

Also, you must configure the plugin to talk to the Schedules Direct beta server for the next few days or so as production hasn't been upgraded yet. So you will have to register your headends on the beta server after upgrading today then when you're told to switch back to the production server, you will (likely) have to register your headends again.

So, today's instructions:

1) Upgrade plugin to new version
2) Go to plugin config screen
3) Change the 'Base URL for sdjson' value to http://23.21.174.111
4) Change the 'Max Schedule Request' value from 1000 to 150
5) Change the 'Max Program Request' value from 50000 to 10000

When production is upgraded, we'll let you know and you can reverse these changes. It is important to modify all the settings as the beta server isn't nearly as beefy as the production server so you must reduce the load.

EDIT: Alternatively, you can wait until production is upgraded at SD and continue to use 0.4.2 until then. In that case, do not upgrade to the new version when notified by Sage.

[rdefino]

HI Slugger,

Thanks for the hard work.

One question, I've been noticing the "no data" on different channels in the line up once and a while. Not a lot. But last night, 9pm est on comcast 804 there was no data just for the 9pm time slot.

Is this upgraded going to help with that, or is that an entirely different problem?

thanks