PlugIn: YouTube (2015)

[tmiranda]

Originally Posted by EnterNoEscape

No problem. It will probably be tomorrow. I'm getting some target invocation errors related to the plugin that implies that for some reason the .jar didn't load, so I'll be restarting SageTV after tonight's recordings to see if it fixes itself. I don't think this means anything is wrong with the plugin, but in case you're interested, here's the logs from that.

Edit: It looks like the plugin isn't loading. I uploaded another attachment with the failure log.

I can't tell what went wrong from those logs

Is this happening all the time or was this just a one off? It looks like it loads correctly but then when the UI tries to access the methods all hell breaks loose.

[EnterNoEscape]

Originally Posted by tmiranda

I can't tell what went wrong from those logs

Is this happening all the time or was this just a one off? It looks like it loads correctly but then when the UI tries to access the methods all hell breaks loose.

I fired up another SageTV instance that I have and it's working fine. It's just not working in the Docker container. I'll play around a little and see if I can find anything else. If I'm seeing this, I'm sure someone else will eventually.

[EnterNoEscape]

Originally Posted by tmiranda

I can't tell what went wrong from those logs

Is this happening all the time or was this just a one off? It looks like it loads correctly but then when the UI tries to access the methods all hell breaks loose.

I did try restarting everything, but the problem persists.

The problem seems to happen only 9.0.13.536 on Linux. The Windows version of SageTV 9.0.13.536 does not have this problem. Rolling the container back to 9.0.12.506 fixes the problem, but that's not an option for anyone who as upgraded because the database was upgraded too. I wonder who would implement changes to the database? Anyway that doesn't appear to a part of the problem.

I noticed that the STV being used by the Linux install is different from the one in the Windows installer, so I tried using that STV on Windows. Things were fine.

I noticed that Sage.jar had different time stamps and otherwise looked the same, but just in case, I ran that on Windows too. Everything was still working.

Either the Windows build is off by a commit that really broke something or the Linux build has a consistency problem. I should probably open an issue on GitHub.

Edit: What's so bizarre to me is that the function that seems to be broken I believe is a part of the standard Java libraries and you would think the problem would travel with Sage.jar. The way it's breaking would seem to imply we are not converting something correctly anymore. I can tell YouTube.jar is loading, but for some reason it can't be used through reflection.

[tmiranda]

Originally Posted by EnterNoEscape

I did try restarting everything, but the problem persists.

The problem seems to happen only 9.0.13.536 on Linux. The Windows version of SageTV 9.0.13.536 does not have this problem. Rolling the container back to 9.0.12.506 fixes the problem, but that's not an option for anyone who as upgraded because the database was upgraded too. I wonder who would implement changes to the database? Anyway that doesn't appear to a part of the problem.

I noticed that the STV being used by the Linux install is different from the one in the Windows installer, so I tried using that STV on Windows. Things were fine.

I noticed that Sage.jar had different time stamps and otherwise looked the same, but just in case, I ran that on Windows too. Everything was still working.

Either the Windows build is off by a commit that really broke something or the Linux build has a consistency problem. I should probably open an issue on GitHub.

Edit: What's so bizarre to me is that the function that seems to be broken I believe is a part of the standard Java libraries and you would think the problem would travel with Sage.jar. The way it's breaking would seem to imply we are not converting something correctly anymore. I can tell YouTube.jar is loading, but for some reason it can't be used through reflection.

What can I do to help you debug?

[EnterNoEscape]

Originally Posted by tmiranda

What can I do to help you debug?

Unfortunately I'm not sure you can do much. This looks like it might be a core problem and nothing to do with your plugin other than it exposes the issue.

[EnterNoEscape]

Actually, maybe you can help me out by telling me what's happening here that's creating an NPE.

Thu 1/26 19:56:10.766 [Fork-TMIR-679620@363af177] java.lang.NullPointerException
Thu 1/26 19:56:10.766 [Fork-TMIR-679620@363af177] java.util.Properties$LineReader.readLine(Properties.java:434)
Thu 1/26 19:56:10.766 [Fork-TMIR-679620@363af177] java.util.Properties.load0(Properties.java:353)
Thu 1/26 19:56:10.766 [Fork-TMIR-679620@363af177] java.util.Properties.load(Properties.java:341)
Thu 1/26 19:56:10.766 [Fork-TMIR-679620@363af177] tmiranda.youtube.Search.getKeyFromProperties(Search.java:109)
Thu 1/26 19:56:10.766 [Fork-TMIR-679620@363af177] tmiranda.youtube.Videos.<init>(Videos.java:45)

Edit: I was looking at the source code from the Java side of things and it looks like the only way you're going to get that kind of exception is if InputStream is null.

void load(InputStream) throws IOException

[tmiranda]

Here is the method:

    static String getKeyFromProperties() {
        
        // Read the developer key from the properties file.
        Properties properties = new Properties();
        
        try {
            InputStream in = Search.class.getResourceAsStream("/" + PROPERTIES_FILENAME);
            properties.load(in);

        } catch (IOException e) {
            Log.getInstance().write(Log.LOGLEVEL_ERROR, "Search: There was an error reading " + PROPERTIES_FILENAME + ": " + e.getCause() + " : " + e.getMessage());
            return null;
        }
        
        return properties.getProperty("youtube.apikey");
    }

The NPE is being thrown by line 109: "properties.load(in)".

That doesn't make a lot of sense to me because "properties" can't be null. The file it's reading is youtube.properties located in the install directory.

[jusjoken]

Originally Posted by EnterNoEscape

I noticed that the STV being used by the Linux install is different from the one in the Windows installer, so I tried using that STV on Windows. Things were fine.

This comment concerned me so I installed the latest windows version and the latest Linux version and did a file compare of the 2 SageTV7.xml files and they are identical in the compare tool.

Interesting that the file SIZES are NOT the same but when using a compare tool for the contents they are a match.

Not that this has anything to do with this issue but I at least feel better that the installers are using the same source

k

[EnterNoEscape]

Originally Posted by jusjoken

This comment concerned me so I installed the latest windows version and the latest Linux version and did a file compare of the 2 SageTV7.xml files and they are identical in the compare tool.

Interesting that the file SIZES are NOT the same but when using a compare tool for the contents they are a match.

Not that this has anything to do with this issue but I at least feel better that the installers are using the same source

k

You know what, I think it's the line endings. I didn't do a binary comparison; the sizes were just different.

[EnterNoEscape]

Originally Posted by tmiranda

Here is the method:

    static String getKeyFromProperties() {
        
        // Read the developer key from the properties file.
        Properties properties = new Properties();
        
        try {
            InputStream in = Search.class.getResourceAsStream("/" + PROPERTIES_FILENAME);
            properties.load(in);

        } catch (IOException e) {
            Log.getInstance().write(Log.LOGLEVEL_ERROR, "Search: There was an error reading " + PROPERTIES_FILENAME + ": " + e.getCause() + " : " + e.getMessage());
            return null;
        }
        
        return properties.getProperty("youtube.apikey");
    }

The NPE is being thrown by line 109: "properties.load(in)".

That doesn't make a lot of sense to me because "properties" can't be null. The file it's reading is youtube.properties located in the install directory.

So Search.class.getResourceAsStream("/" + PROPERTIES_FILENAME); never returns null?

I have a strange feeling that between the two versions of SageTV, a variable or what is considered the current directory changed.

[EnterNoEscape]

I found the problem. It was the change that added lucene ahead of all of the other JARs.

A period (.) was removed that changes where the JAR thinks it's loaded. I have a feeling it was looking for the file on the root of the filesystem. I put the period back manually for my own installation and now suddenly everything is working. I'll get a commit in later today if someone doesn't beat me to it.

https://github.com/google/sagetv/com...2994663dcec9f4

[jusjoken]

Originally Posted by EnterNoEscape

I found the problem. It was the change that added lucene ahead of all of the other JARs.

A period (.) was removed that changes where the JAR thinks it's loaded. I have a feeling it was looking for the file on the root of the filesystem. I put the period back manually for my own installation and now suddenly everything is working. I'll get a commit in later today if someone doesn't beat me to it.

https://github.com/google/sagetv/com...2994663dcec9f4

The removal of the "." was mentioned here in case you were wondering...

https://forums.sagetv.com/forums/sho...8&postcount=14

k

[EnterNoEscape]

Originally Posted by jusjoken

The removal of the "." was mentioned here in case you were wondering...

https://forums.sagetv.com/forums/sho...8&postcount=14

k

Thanks for the reference. I wonder if this breaks any other plugins that are making the same assumption as this one. What would be the best approach would be for this plugin to be updated so that it's not so dependent on that configuration. This does make me kind of wonder if anything else in SageTV isn't working correctly due to that change.

[tmiranda]

Thanks for discovering the issue. I don't know about other developers but when working with files in plugins I have always assumed the root is the install directory. That's what worked for me from the start and I've stuck with it out of habit.

[EnterNoEscape]

When I develop in any programming language I try to use the currently accepted practices for the OS in question. Since this will vary between different platforms, I create variables that are always used when referencing specific categories of files. This usually ends up being configuration and logging. I also set a variable that will always be the same directory as the program executable or equivalent so I can access anything within that structure reliably.

In the case of SageTV, it's following the really old unstructured way that Windows used to do things. So in this case, I would still define all of those individually for one day when it might actually be helpful. It's just that today they would all consistently point to paths relative to Sage.jar.

[stuckless]

Keep in mind, code like,

Search.class.getResourceAsStream("/" + PROPERTIES_FILENAME);

make no assumptions about install directories or root directories, because neither of those 2 things matter.

Assuming PROPERTIES_FILENAME is "myprop.properties"...
The above code, basically uses the Java class loader and starts to look for an entry "/myprop.properties" in the ROOT of EACH classpath entry.

So, consider the following... if your classpath was "my.jar:sage.jar:.", then it would first look in my.jar, then sage.jar, and then the current working directory.

So, depending on "where" "." is located in the classpath you could different results... ie, if my.jar had a "myprop.properties", then that's the properties you'd get, not the one in the current working directory.

Typically the ONLY time to use the above code, is if you package a properties inside your JAR file, and you want to get it there.

If you are loading a properties from an actual file on the filesystem, then it's better to use

InputStream = new FileInputStream(new File("myprop.properties"))

Which makes the assumption that you are actually loading the properties from a file in the curent working directory.

I have no issues if someone wants to put the "." back in the classpath, but, I do think that original code was misusing it anyways, and likely, it should have been using a FileInputStream if it really did mean to load something in the current working directory.

[tmiranda]

Originally Posted by stuckless

Keep in mind, code like,

Search.class.getResourceAsStream("/" + PROPERTIES_FILENAME);

make no assumptions about install directories or root directories, because neither of those 2 things matter.

Assuming PROPERTIES_FILENAME is "myprop.properties"...
The above code, basically uses the Java class loader and starts to look for an entry "/myprop.properties" in the ROOT of EACH classpath entry.

So, consider the following... if your classpath was "my.jar:sage.jar:.", then it would first look in my.jar, then sage.jar, and then the current working directory.

So, depending on "where" "." is located in the classpath you could different results... ie, if my.jar had a "myprop.properties", then that's the properties you'd get, not the one in the current working directory.

Typically the ONLY time to use the above code, is if you package a properties inside your JAR file, and you want to get it there.

If you are loading a properties from an actual file on the filesystem, then it's better to use

InputStream = new FileInputStream(new File("myprop.properties"))

Which makes the assumption that you are actually loading the properties from a file in the curent working directory.

I have no issues if someone wants to put the "." back in the classpath, but, I do think that original code was misusing it anyways, and likely, it should have been using a FileInputStream if it really did mean to load something in the current working directory.

Yes, I see I needed to change that. I actually changed it to the following which seems to work. Is this OK? (PROPERTIES_FILENAME is "youtube.properties")

            InputStream in = new FileInputStream(PROPERTIES_FILENAME);     
            properties.load(in);

[stuckless]

Originally Posted by tmiranda

Yes, I see I needed to change that. I actually changed it to the following which seems to work. Is this OK? (PROPERTIES_FILENAME is "youtube.properties")

            InputStream in = new FileInputStream(PROPERTIES_FILENAME);     
            properties.load(in);

That code is fine, if your expectation is that you want to load the properties in the sagetv root. While getResourceAsStream will search multiple locations for the file, your above code will only search the current work directory (which is what you likely intended before).

One certainty that is likely never going to change with SageTV is the new File(".") will always return the current working directory (ie, sagetv will always be started from it's root.).

In Phoenix, everything is relative to PHOENIX_HOME and was designed that way, so that "phoenix" could live anywhere, but sagetv was never designed with a "SAGE_HOME" concept, so it relies on current working directory being the sagetv server root directory.

That's a long explanation for "yes, the above code will do what you want"

[tmiranda]

Originally Posted by stuckless

That code is fine, if your expectation is that you want to load the properties in the sagetv root. While getResourceAsStream will search multiple locations for the file, your above code will only search the current work directory (which is what you likely intended before).

One certainty that is likely never going to change with SageTV is the new File(".") will always return the current working directory (ie, sagetv will always be started from it's root.).

In Phoenix, everything is relative to PHOENIX_HOME and was designed that way, so that "phoenix" could live anywhere, but sagetv was never designed with a "SAGE_HOME" concept, so it relies on current working directory being the sagetv server root directory.

That's a long explanation for "yes, the above code will do what you want"

Thanks Sensei

[tmiranda]

A new version is available in the repository. This version fixes the bug that shows up in the latest builds of SageTV and it incorporates the latest youtube-dl.