Plex Scanner and BMT Agent for SageTV

[Fuzzy]

Originally Posted by rerooks

I didn't realize that. Normally the first post of a thread is updated. Is it github now?

Yeah, the original author is no longer around - wildgoose has effectively taken ownership of the project, but he can't alter the first post. He's got a link in his signature to the current build and instructions (which yes, are on github)

[rerooks]

Great. I found it and the install is much easier than before.

[Opus4]

Originally Posted by Fuzzy

Yeah, the original author is no longer around - wildgoose has effectively taken ownership of the project, but he can't alter the first post. He's got a link in his signature to the current build and instructions (which yes, are on github)

I added a link to the new info in the first post.

Andy

[Fuzzy]

Originally Posted by Opus4

I added a link to the new info in the first post.

Andy

Thank you Andy! Likely save a lot of hassle.

[z3r0]

I have V9 installed on Linux and I'm getting the following in my log when I try to refresh my SageTV Library within Plex:

2016-02-22 01:22:59,740| DEBUG | openUrl: http://sage:frey@localhost:8080/sage...s&encoder=json
2016-02-22 01:22:59,751| DEBUG | getMediaFileForName(Zombieland-904228-0.ts): not found
2016-02-22 01:22:59,751| ERROR | No media info from SageTV: Zombieland-904228-0.ts
2016-02-22 01:22:59,751| INFO | Total: 0 of 665 added to mediaList

I get this same error 665 times in the log, for each file it finds in the Library directory.

If I try to manually call the GetMediaFileForName via url, I get the following error output:

URL:

http://sage:frey@X.X.X.X:8080/sagex/api?c=plex:GetMediaFileForName&1=Zombieland-904228-0.ts&encoder=json

Output:

{"sagexVersion":"7.1.9.1","exception":"java.lang.RuntimeException: javax.script.ScriptException: TypeError: sagex.api.MediaFileAPI@5658404d has no such function \"GetMediaFiles\" in <eval> at line number 46\n\tat<br> sagex.remote.api.ApiHandler.callService(ApiHandler.java:200)\n\tat sagex.remote.api.ApiHandler.handleRequest(ApiHandler.java:140)\n\tat sagex.remote.SagexServlet.doPost(SagexServlet.java:87)\n\tat sagex.remote.SagexServlet.doGet(SagexServlet.java:41)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:707)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:820)\n\tat org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)\n\tat org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:389)\n\tat org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)\n\tat org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)\n\tat org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)\n\tat org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)\n\tat org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)\n\tat org.mortbay.jetty.handler.rewrite.RewriteHandler.handle(RewriteHandler.java:230)\n\tat org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)\n\tat org.mortbay.jetty.Server.handle(Server.java:326)\n\tat org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)\n\tat org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)\n\tat org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:539)\n\tat org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)\n\tat org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)\n\tat org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)\n\tat org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520)\nCaused by: javax.script.ScriptException: TypeError: sagex.api.MediaFileAPI@5658404d has no such function \"GetMediaFiles\" in <eval> at line number 46\n\tat jdk.nashorn.api.scripting.NashornScriptEngine.throwAsScriptException(NashornScriptEngine.java:467)\n\tat jdk.nashorn.api.scripting.NashornScriptEngine.invokeImpl(NashornScriptEngine.java:389)\n\tat jdk.nashorn.api.scripting.NashornScriptEngine.invokeFunction(NashornScriptEngine.java:190)\n\tat sagex.remote.api.ServiceFactory.callService(ServiceFactory.java:83)\n\tat sagex.remote.api.ApiHandler.callService(ApiHandler.java:198)\n\t... 22 more\nCaused by: <eval>:46 TypeError: sagex.api.MediaFileAPI@5658404d has no such function \"GetMediaFiles\"\n\tat jdk.nashorn.internal.runtime.ECMAErrors.error(ECMAErrors.java:57)\n\tat jdk.nashorn.internal.runtime.ECMAErrors.typeError(ECMAErrors.java:213)\n\tat jdk.nashorn.internal.runtime.ECMAErrors.typeError(ECMAErrors.java:185)\n\tat jdk.nashorn.internal.runtime.ECMAErrors.typeError(ECMAErrors.java:172)\n\tat jdk.nashorn.internal.runtime.linker.NashornBottomLinker.linkBean(NashornBottomLinker.java:106)\n\tat jdk.nashorn.internal.runtime.linker.NashornBottomLinker.getGuardedInvocation(NashornBottomLinker.java:71)\n\tat jdk.internal.dynalink.support.CompositeGuardingDynamicLinker.getGuardedInvocation(CompositeGuardingDynamicLinker.java:124)\n\tat jdk.internal.dynalink.support.LinkerServicesImpl.getGuardedInvocation(LinkerServicesImpl.java:154)\n\tat jdk.internal.dynalink.DynamicLinker.relink(DynamicLinker.java:253)\n\tat jdk.nashorn.internal.scripts.Script$Recompilation$3$1499A$\\^eval\\_.GetMediaFileForName(<eval>:46)\n\tat jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:642)\n\tat jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:228)\n\tat jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:393)\n\tat jdk.nashorn.api.scripting.ScriptObjectMirror.callMember(ScriptObjectMirror.java:199)\n\tat jdk.nashorn.api.scripting.NashornScriptEngine.invokeImpl(NashornScriptEngine.java:383)\n\t... 25 more\n","error":"javax.script.ScriptException: TypeError: sagex.api.MediaFileAPI@5658404d has no such function \"GetMediaFiles\" in <eval> at line number 46"}

My installation is probably an outlier compared to most (SmartOS Linux zone), but everything else is working fine. Perhaps this code or some dependent plug-in doesn't quite support SageTV V9 correctly? That or I feel like I could be missing some simple config value somewhere.

FYI: I have BMT, sagex, Phoenix, etc plugins installed. JVM is 1.8.

[wildgoose]

Looks like it's failing on this line:

var allMedia = MediaFileAPI.GetMediaFiles();

https://github.com/ai7/sagetv-for-pl...es/plex.js#L46

GetMediaFiles is a standard SageTV API:
http://download.sage.tv/api/sage/api...GetMediaFiles()

I remember there were some issues with JRE 1.8 being reported related to the jetty web server (which sagex/plex uses).

Not sure if those got addressed or not. Can you try with JRE 1.7 to see if it helps?

[fresnoboy]

Guys, thanks for all the work here. I am about to try and integrate this into my SageTV instance, but wanted to know if the new plugin supports auto-skip during sage playback if I have EDL files associated with the recordings?

Also, how does Plex perform with a LOT of recordings? I have about 5000 recordings in about 30 TB of storage...

Thx
Mike

[Fuzzy]

Originally Posted by fresnoboy

Guys, thanks for all the work here. I am about to try and integrate this into my SageTV instance, but wanted to know if the new plugin supports auto-skip during sage playback if I have EDL files associated with the recordings?

Also, how does Plex perform with a LOT of recordings? I have about 5000 recordings in about 30 TB of storage...

Thx
Mike

No, Plex doesn't support any sort of skipping like that, and doesn't even support skipping by chapters, so there's not really a workaround either.

[Taddeusz]

After having some problems and doing some troubleshooting I ended up having to redo my SageTV configuration. Ever since then it won't load the fanart and I've been seeing a lot of these warnings in the log:

WARNING (logkit:19) - getFanArt: <html> detected in fanart response, not fanart

When I manually browse to one of the failed links it is an HTML page with an img tag linked to the image. Do I have something configured wrong?

I'm also seeing a lot of these errors:

ERROR (logkit:22) - openUrl: failed on: http://localhost:32400/:/scrobble?key=158150&identifier=com.plexapp.plugins.library: ('http error', 401, 'Unauthorized', <httplib.HTTPMessage instance at 0x02134B20>)

I don't know why it would be getting a 401 error on Plex itself?

[wildgoose]

Originally Posted by Taddeusz

After having some problems and doing some troubleshooting I ended up having to redo my SageTV configuration. Ever since then it won't load the fanart and I've been seeing a lot of these warnings in the log:

WARNING (logkit:19) - getFanArt: <html> detected in fanart response, not fanart

When I manually browse to one of the failed links it is an HTML page with an img tag linked to the image. Do I have something configured wrong?

I'm also seeing a lot of these errors:

ERROR (logkit:22) - openUrl: failed on: http://localhost:32400/:/scrobble?key=158150&identifier=com.plexapp.plugins.library: ('http error', 401, 'Unauthorized', <httplib.HTTPMessage instance at 0x02134B20>)

I don't know why it would be getting a 401 error on Plex itself?

Hmm, interesting. The fanart is obtained from sagex. Looking at the code, looks like it's expecting an image directly from the response:

https://github.com/ai7/sagetv-for-pl.../sagex.py#L282

So you are saying instead you get an <HTML> response and inside there's an <IMG> link? I recall in my testing HTML response usually indicated an error.. Can you provide the HTML response text?

I am not sure why PLEX would throw an unauthorized from localhost. Usually if one is querying from a different host then a special setting needs to be enabled (security wise), but localhost should by default work. Something might have changed recently. I am still running v7 windows with an older PMS. I'll try the latest PMS in the next few days to see if I encounter any issues.

[Taddeusz]

Originally Posted by wildgoose

Hmm, interesting. The fanart is obtained from sagex. Looking at the code, looks like it's expecting an image directly from the response:

https://github.com/ai7/sagetv-for-pl.../sagex.py#L282

So you are saying instead you get an <HTML> response and inside there's an <IMG> link? I recall in my testing HTML response usually indicated an error.. Can you provide the HTML response text?

I am not sure why PLEX would throw an unauthorized from localhost. Usually if one is querying from a different host then a special setting needs to be enabled (security wise), but localhost should by default work. Something might have changed recently. I am still running v7 windows with an older PMS. I'll try the latest PMS in the next few days to see if I encounter any issues.

Ok, I got this figured out. It was a configuration problem. While troubleshooting I went into BMT and noticed that the fan art folder setting was wrong. I corrected the setting and everything seems to be working now.

[wildgoose]

Originally Posted by Taddeusz

I corrected the setting and everything seems to be working now.

Great!

[Taddeusz]

As it turns out all this was because I apparently accidentally approved the upgrade to Java 8 when I was remoted in the other day. I guess that's what I get for swapping between remote sessions and not realizing which system I was in when it popped up. No matter how many times I try disabling checking for Java updates it keeps popping up the message that there's an update available. Very frustrating.

[EnterNoEscape]

Originally Posted by Taddeusz

As it turns out all this was because I apparently accidentally approved the upgrade to Java 8 when I was remoted in the other day. I guess that's what I get for swapping between remote sessions and not realizing which system I was in when it popped up. No matter how many times I try disabling checking for Java updates it keeps popping up the message that there's an update available. Very frustrating.

I build Windows images for mass deployment at work and used the commands below in my scripting process to remove all notifications about Java updates because we control them via SCCM. I now use something similar to this in PowerShell, but I figured it would be easier to provide for the command prompt. This will also remove the Java update tab from the Control Panel applet. I offer no guarantee this will get the prompts to go away, but it has worked well for me in an enterprise environment. You may need to kill the Java update process before doing this since it might put the keys back in place.

REG DELETE "HKLM\SOFTWARE\JavaSoft\Auto Update" /va /f
REG DELETE "HKLM\SOFTWARE\JavaSoft\Java Update" /va /f
REG DELETE "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run" /v "SunJavaUpdateSched" /f
REG DELETE "HKLM\SOFTWARE\Wow6432Node\JavaSoft\Auto Update" /va /f
REG DELETE "HKLM\SOFTWARE\Wow6432Node\JavaSoft\Java Update" /va /f
REG DELETE "HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Run" /v "SunJavaUpdateSched" /f

[Taddeusz]

I figured out the Java updates thing. Apparently making changes in the Java Control Panel requires Administrator rights. When you run it from the Windows Control Panel it doesn't run with elevated rights so any changes being made don't get saved. You have to manually run it as an administrator and make the changes there. Flipping stupid. Apparently this has been a "bug" for quite some time. Really shows where Oracle's Java priorities lie when they won't take care of something so basic as this.

[wildgoose]

Originally Posted by Taddeusz

I figured out the Java updates thing. Apparently making changes in the Java Control Panel requires Administrator rights. When you run it from the Windows Control Panel it doesn't run with elevated rights so any changes being made don't get saved.....

Yep, I ran into the same issue a while ago and found the same solution.

[fresnoboy]

Folks, I got the Plex plugin integrated with BMT and it's very very nice. Thank you everyone!!!

A few issues though:

1) The plugin takes a very very long time to plow through all the recordings. I have about 9500 recordings, and it can take more than a day to go through them.

2) Often times when doing a scan, SageTV will lock up and BMT reports Sage has run out of memory. I have the heapsize set to 400 in the registry, and run a very clean configuration, so I don't exactly know what to do about this.

3) The FanArt varies. In movies, it looks like if BMT doesn't have fanart, Plex fills in it with a very high res version compared to the one BMT uses. Is it possible for Plex to provide fanart for movies at least instead of BMT?

4) When SageTV runs out of memory, it appears that plex forgets about all the recordings, as when I go to Movies or TV shows, nothing shows up. I was assuming as long as it could get to the file, plex would not try and do a metadata lookup, but it appears there is some linkage here.

5) Some shows have odd metadata re: seasons. For some shows, Plex uses a year (like 2014), and for others a season (like season 12). Why are these not consistent?

Thanks!
Mike

[fresnoboy]

Well, turns out I actually did not have the server using 1GB of heap as I had thought. Its fixed now, and everything is working much better.

On the other issues, I think the reason I have low res fanart is because Sage fetched that fanart several years ago, and the recordings of sfavorites that I just started to record have very good fanart. Should I just delete all the current fanart and tell BMT to recreate it, and that would have new fanart for everything?

Also, in Plex, many favorites show up as 2 or even 3 discrete shows at the TV show level of Plex. It's not even a season based difference. They all show up as one favorite in Sage recordings. Can someone explain why this might be happening, and how I can remedy it?

Thanks again for all the work on this plugin. It's made consuming shows from my library 100x easier, esp with automatic transcode and download to tablets and phones and Chromecast support.

Thanks,
Mike

[wildgoose]

I am glad you are able to resolve the issues!

Originally Posted by fresnoboy

1) The plugin takes a very very long time to plow through all the recordings. I have about 9500 recordings, and it can take more than a day to go through them.

That's an impressive collection! Still more than a day seems a little too much. The scanner should've finished scanning the folders relatively quickly. You can check its status in the sageplex_scanner.log file. Once that's done, the BMT agent starts to download and fill in the metadata info. This can be observed in com.plexapp.agents.bmtagenttvshows.log. This takes like a few minutes on mine which has about 100+ show files.

While the BMT agent is running the shows should already be visible in PLEX, but without the fanart, etc..

If it's taking too long, you can check the logs to see whether there's any errors or exceptions.

Originally Posted by fresnoboy

5) Some shows have odd metadata re: seasons. For some shows, Plex uses a year (like 2014), and for others a season (like season 12). Why are these not consistent?

Generally, if a show has season information from SageTV, like season 12, then it will be used. However, if it doesn't, then the plugin will manually assign it a season based on the year, so it'll be something like Season 2014. So for shows that shows up in PLEX with season YEAR, it means SageTV did not report a season number for those shows.

This tagging information can be fixed/updated on the SageTV side manually. I've done so in the past with some of my shows.. (don't have the exact URL you need to browse to to edit this info at the moment...).

Also, in Plex, many favorites show up as 2 or even 3 discrete shows at the TV show level of Plex. It's not even a season based difference. They all show up as one favorite in Sage recordings. Can someone explain why this might be happening, and how I can remedy it?

Some aspect of the metadata is causing the plugin to categorize it into 2 different shows. It'll help if we have the metadata that SageTV stores for those episodes so we can look at them and figure out why.

Let me look into how we can get this data dump. You can do it with sageplex_sync but it's not exactly straightforward, especially for large libraries.

[fresnoboy]

Thanks!

One of the other problems I had was that I had was that I had the plugin run every time a directory change happened. Well, given there are literally thousands of shows in each directory, and the start of a recording would cause the plugin to start again, there seemed to be issues with being able to complete a scan successfully.

I changed it to run once every 6 hours, and now the scanning seems to find all the shows and movies properly.

For the Movies, Plex seems to find higher quality Fanart than BMT does. This is probably not a plugin question, but have you found a way to have plex do all the fanart or BMT to use a higher res source? I can fix it manually in plex, but I'd rather like something that was automatic.

Is there a log file I can look at that would have the times to complete each scan, etc...? The plex instance is running in a different VM than SageTV does on a different hardware (I have 2 2xE5-2670's that my guests run on - the NAS runs on bare metal on a separate system), with tons of RAM and 8 cores, so it's should be a pretty fast platform for doing the processing.

Thanks again!
Mike