Plugin: Schedules Direct EPG (not for v9)

[Slugger]

Fix release with some key fixes:

* Fix issue where using multiple SD lineups on multiple Sage inputs would result in one lineup's channel always having no data

* Add support for various additional metadata values (Dolby, letterboxed, etc, etc)

* Add support for SD's new movie star rating format

* Add support for auto backing up of local cache

* Add support for not attempting a network pull of EPG data from SD; just reproduce existing cache file

* Various other fixes and tweaks

This is the version all users of this plugin should be on.

Note: I've been told that the old 20130709 API version at SD will be shut down within the next week. This means v0.0.5.x of this plugin will stop working when SD shuts down the old API service.

[BobbyDing]

OK, here are the latest logs. Last night after posting I removed all the plugins and re installed the plugins. After checking the plugin settings I went to the sagetv web interface, selected configure lineups, at which point it errors out. That's as far as I get. Logs attached.

Thanks!

Bobby

P.S. I ran the plugin update first thing before getting these logs.

[Slugger]

[[[ START REQUEST: Fri May 16 01:51:14 EDT 2014
>>>target: https://json.schedulesdirect.org/20131021/lineups
>>>verb: GET
>>>req_headers:
	token: c233e531ae199db47764b206f2d02de2
<<<resp_status: HTTP/1.1 400 Bad Request
<<<resp_headers:
	Content-Type: application/json;charset=UTF-8
	Date: Fri, 16 May 2014 05:49:48 GMT
	Schedulesdirect-Serverid: AWS-SD-web.1
	Server: Apache/2.2.22 (Ubuntu)
	Vary: Accept-Encoding
	X-Powered-By: PHP/5.4.9-4ubuntu2.4
	Connection: keep-alive
*** REQUEST FAILED! ***
HTTP request did not return rc=200!
END REQUEST: Fri May 16 01:51:15 EDT 2014]]]

So this is the problem. It appears to be an issue with SD _or_ my guess is you have no lineups registered in your account, but I can't tell from the logs because the error returned from SD appears to contain no error text.

You should open an issue ticket the SD web site and provide the info above, they should be able to get more details about the issue from their logs.

[rkulagow]

Here's what's on the server-side:

generateJSONresponse:json is {"response":"NO_LINEUPS","code":4102,"serverID":"AWS-SD-web.1","message":"No lineups have been added to this account.","datetime":"2014-05-16T05:49:48Z"}

Have you added your lineups to the new API? The new API lives on a much more permanent set of hosts, which is why there's a new URL, and that means that you'll also need to go through the add lineup step.

I will work with slugger to see why you didn't get the error response.

[Slugger]

Looks like a bug was introduced in the latest client api changes I delivered last week. I don't think you'll be able to register the lineups to your account using the sdepg tools until I fix it. Wait for the next sdepg plugin update, it will contain the fix. I'll try to get something out today, but no promises.

EDIT: I'm able to reproduce the issue, it's definitely an issue with my code. Again, I'll try to get a fix out today, but might be sometime on the weekend before I can get to it.

[BobbyDing]

Thanks Guys. No hurry on my end. I appreciate all you do.

Bobby

[Slugger]

So here's the quick fix. I need to actually go back and rework the error handling for a "proper" fix, but this will get people up and running asap in the meantime.

So replace the entire contents of SageTV\webserver\groovy\sdjson\headends.gsp with the content below. You'll then be able to proceed with the registration of lineups to your account.

<%
/*
 *      Copyright 2013-2014 Battams, Derek
 *       
 *       Licensed under the Apache License, Version 2.0 (the "License");
 *       you may not use this file except in compliance with the License.
 *       You may obtain a copy of the License at
 *
 *          http://www.apache.org/licenses/LICENSE-2.0
 *
 *       Unless required by applicable law or agreed to in writing, software
 *       distributed under the License is distributed on an "AS IS" BASIS,
 *       WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *       See the License for the specific language governing permissions and
 *       limitations under the License.
 */
	import org.schedulesdirect.api.exception.InvalidHttpResponse

	def clnt = request.getAttribute('clnt')
	def mub
	def lineups
	try { lineups = clnt.lineups } catch(InvalidHttpResponse e) { lineups = [] }
%>
<h2>Configured Headends</h2>
<form action="index.gsp" method="post">
	<input type="hidden" name="c" value="rmhe" />
	<input type="hidden" name="m" value="headends" />
	<table>
		<tr>
			<th><input type="checkbox" name="_" id="headends" /></th>
			<th>ID</th>
			<th>Description</th>
		</tr>
		<% mub = new groovy.xml.MarkupBuilder(out); lineups.each { he ->
			mub.tr {
				td { input(type:'checkbox', name:'heid', value:"lineups/$he.id", class:'regId') }
				td he.id
				td "$he.name $he.location"
			}
		} %>
	</table>
	<input type="submit" name="submit" value="Remove Selected" />
</form>

<h2>Add Headend</h2>
<div>
	<span>Enter zip/postal code:</span>
	<span><input type="text" name="zip" id="srchInput" /></span>
</div>
<div>
	<span>Enter ISO country code:</span>
	<span><input type="text" name="country" id="srchCountry" /></span>
	<span><input type="button" id="search" value="Search" /></span>
</div>
<form action="index.gsp" method="post" id="addFrm" style="visibility: hidden;">
	<input type="hidden" name="c" value="addhe" />
	<input type="hidden" name="m" value="headends" />
	<table>
		<thead>
			<tr>
				<th><input type="checkbox" name="_" id="newHeadends" /></th>
				<th>ID</th>
				<th>Description</th>
			</tr>
		</thead>
		<tbody id="srchResults">

		</tbody>
	</table>
	<input type="submit" name="submit" value="Register Selected" />
</form>
<script>
<!--
\$(document).ready(function() {
	\$('#headends').change(function() {
		var val = \$(this).prop('checked');
		\$('.regId').each(function() {
			\$(this).prop('checked', val);
		});
	});
	\$('#newHeadends').change(function() {
		var val = \$(this).prop('checked');
		\$('.newId').each(function() {
			\$(this).prop('checked', val);
		});
	});
	\$('#search').click(function() {
		\$(this).attr('disabled', true);
		\$(this).attr('value', 'Searching...');
		\$.ajax({
			url: '/sage/sdjson/ajax.groovy',
			context: \$(this),
			data: {'z': \$('#srchInput').val(), 'c': 'search', 'i': \$('#srchCountry').val()},
			success: function() {
				\$('#srchResults').html(arguments[0]);
				\$('#addFrm').css('visibility', 'visible');
			
			},
			error: function() {
				alert('Search failed!  Please try again.');
			},
			complete: function() {
				\$(this).attr('disabled', false);
				\$(this).attr('value', 'Search');
			}
		});
	});
});
-->
</script>

[BobbyDing]

Thanks Slugger, I'll try it when I get home tonight.

Bobby

[rkulagow]

As discussed here http://forums.sagetv.com/forums/showthread.php?t=61440 please let me and slugger know.

It may be possible to address this server side (by creating -1, -2, etc channels for DirecTV), but I don't have DirecTV, so we need to gather some data from affected users / beta testers.

[rdefino]

So I updated to the latest version and now I'm seeing below in my sdepg.log file.

Could Iget some direction on what to check. I saw this"reconfigure your input sources" but not sure what t means.

thanks



... 29 more
Caused by: java.net.URISyntaxException: Illegal character in path at index 16: file:/C:/Program Files (x86)/SageTV/SageTV/plugins/sdepg/sdjson.epg
at java.net.URI$Parser.fail(Unknown Source)
at java.net.URI$Parser.checkChars(Unknown Source)
at java.net.URI$Parser.parseHierarchical(Unknown Source)
at java.net.URI$Parser.parse(Unknown Source)
at java.net.URI.<init>(Unknown Source)
at com.sun.nio.zipfs.ZipFileSystemProvider.uriToPath(ZipFileSystemProvider.java:85)
... 50 more
2014-05-15 19:02:12,893 WARN [Plugin]: Failed to handle ButtonClickHandler: 'sdepg/refresh' in class: sagex.epg.schedulesdirect.plugin.Plugin
2014-05-16 09:40:02,316 ERROR [EPGImportPluginSchedulesDirect]: Provider id is unknown, you need to reconfigure your input sources! [1567235544]

[Slugger]

Did you follow the upgrade wiki before applying the upgraded plugin?

[rdefino]

No I didn't . I didn't even mean to install it. I went to the plugin and when to check details and hit the wrong button. Believe me I wasn't happy.

Can I uninstall the plugin, then follow the procedure?

thanks

[Slugger]

Hmm... I think you should just try to proceed with the steps in the wiki, it should be fine. Mind you, I went thru many a vm reverts and retries to get things just right and that wiki was the result. So going slightly out of order isn't ideal, but probably won't hurt anything.

So at this point, you'd delete your video sources in Sage and start at step 3 in the wiki.

If it's any consolation, if you were still running 0.0.5.x, it was going to stop working by the end of this week anyway when SD shuts down the old api server (they've already stopped doing daily epg updates on it) so you were due for an upgrade anyway.

[rdefino]

ok, thanks. I'll give it a shot.

[rdefino]

Hi Slugger, I tried following the steps to update to the new plugin. You told me to start at step 3 since I inadvertently updated the plugin without following the wiki.

I followed the steps and when I went to reconfigure the lineup, I get the error below.





HTTP ERROR 500

Problem accessing /sage/sdjson/index.gsp. Reason:

HTTP request did not return rc=200!
Caused by:

org.schedulesdirect.api.exception.InvalidHttpResponse: HTTP request did not return rc=200!
at org.schedulesdirect.api.json.JsonRequest.submitRaw(JsonRequest.java:201)
at org.schedulesdirect.api.json.JsonRequest.submitForJson(JsonRequest.java:132)
at org.schedulesdirect.api.NetworkEpgClient.getLineups(NetworkEpgClient.java:279)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
at groovy.lang.MetaClassImpl$GetBeanMethodMetaProperty.getProperty(MetaClassImpl.java:3481)
at org.codehaus.groovy.runtime.callsite.GetEffectivePojoPropertySite.getProperty(GetEffectivePojoPropertySite.java:61)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:227)
at SimpleTemplateScript2.run(SimpleTemplateScript2.groovy:32)
at net.sf.sageplugins.webserver.groovy.templates.IncludeTemplateEngine$SimpleTemplate$1.writeTo(IncludeTemplateEngine.java:120)
at net.sf.sageplugins.webserver.groovy.servlets.SageTemplateServlet.service(SageTemplateServlet.java:99)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:389)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
at org.mortbay.jetty.servlet.Dispatcher.include(Dispatcher.java:192)
at javax.servlet.RequestDispatcher$include.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
at SimpleTemplateScript1.run(SimpleTemplateScript1.groovy:158)
at net.sf.sageplugins.webserver.groovy.templates.IncludeTemplateEngine$SimpleTemplate$1.writeTo(IncludeTemplateEngine.java:120)
at net.sf.sageplugins.webserver.groovy.servlets.SageTemplateServlet.service(SageTemplateServlet.java:99)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:389)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.handler.rewrite.RewriteHandler.handle(RewriteHandler.java:230)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:539)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520)

[Slugger]

Follow directons in post 427

[rdefino]

Thanks that worked.

But one more stupid question. I then entered my zip code "01830" and country code "us" and the search fails. Is the country code correct?

thanks again

[Slugger]

No, the country was changed to 3 letters in this new Schedules Direct version. Use 'USA' instead. If I think of it, I'll force the issue via a popup or something on the next release.

[rdefino]

Looks like the epg pull is failing, any thoughts?


2014-05-26 16:34:07,375 INFO [AiringGenerator]: Processed airing generators in 16ms
2014-05-26 16:45:12,154 INFO [ProgramGenerator]: Processed show generators in 0ms
2014-05-26 16:45:12,170 INFO [EpgDownloader]: Requesting 511 channels
2014-05-26 16:45:12,170 INFO [EpgDownloader]: [C:\Program Files (x86)\Java\jre7\bin\java, -Xmx512m, -Dsdjson.fs.capture=C:\Program Files (x86)\SageTV\SageTV\plugins\sdepg\capture\grabber, -jar, C:\Program Files (x86)\SageTV\SageTV\plugins\sdepg\tools\sdjson.jar, --username, rdefino, --password, *****, --user-agent, sagetv-sdepg/0.0.6.6 (Windows 7 x86 6.1), --max-threads, 200, --url, https://json.schedulesdirect.org, --grabber-log-level, INFO, grab, --target, C:\Program Files (x86)\SageTV\SageTV\plugins\sdepg\sdjson.epg, --max-prog-chunk, 50000, --max-sched-chunk, 1000, --stations, C:\Program Files (x86)\SageTV\SageTV\plugins\sdepg\stations.txt, --purge-cache]
2014-05-26 16:45:13,215 ERROR [EpgDownloader]: sdjson download failed! [rc=1]
2014-05-26 16:45:13,215 ERROR [EpgDownloader]: stdout:

2014-05-26 16:45:13,215 ERROR [EpgDownloader]: stderr:
Exception in thread "main" java.lang.IllegalArgumentException: Illegal character in path at index 16: file:/C:/Program Files (x86)/SageTV/SageTV/plugins/sdepg/sdjson.epg
at com.sun.nio.zipfs.ZipFileSystemProvider.uriToPath(ZipFileSystemProvider.java:87)
at com.sun.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:107)
at java.nio.file.FileSystems.newFileSystem(Unknown Source)
at java.nio.file.FileSystems.newFileSystem(Unknown Source)
at org.schedulesdirect.grabber.Grabber.updateZip(Grabber.java:423)
at org.schedulesdirect.grabber.Grabber.execute(Grabber.java:725)
at org.schedulesdirect.grabber.Grabber.main(Grabber.java:832)
Caused by: java.net.URISyntaxException: Illegal character in path at index 16: file:/C:/Program Files (x86)/SageTV/SageTV/plugins/sdepg/sdjson.epg
at java.net.URI$Parser.fail(Unknown Source)
at java.net.URI$Parser.checkChars(Unknown Source)
at java.net.URI$Parser.parseHierarchical(Unknown Source)
at java.net.URI$Parser.parse(Unknown Source)
at java.net.URI.<init>(Unknown Source)
at com.sun.nio.zipfs.ZipFileSystemProvider.uriToPath(ZipFileSystemProvider.java:85)
... 6 more

2014-05-26 16:45:13,215 ERROR [EPGImportPluginSchedulesDirect]: Download of EPG data failed!
java.io.IOException: download failed! [rc=1]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:71)
at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callConstructor(ConstructorSite.java:81)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:190)
at sagex.epg.schedulesdirect.io.EpgDownloader.download(EpgDownloader.groovy:115)
at sagex.epg.schedulesdirect.io.EpgDownloader$download.call(Unknown Source)
at sagex.epg.schedulesdirect.EPGImportPluginSchedulesDirect.doUpdate(EPGImportPluginSchedulesDirect.groovy:348)
at sagex.epg.schedulesdirect.EPGImportPluginSchedulesDirect.updateGuide(EPGImportPluginSchedulesDirect.groovy:214)
at sage.ae.y(Unknown Source)
at sage.a2.e(Unknown Source)
at sage.a9.dK(Unknown Source)
at sage.ae.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2014-05-26 16:45:13,215 INFO [AiringGenerator]: Processed airing generators in 0ms

[Slugger]

Try deleting the sdjson.epg file and forcing another epg update. I vaguely remember seeing that during dev, but the exact cause slips my mind at the moment.