PlugIn: YouTube (2015)

[tmiranda]

Originally Posted by egeller

Thanks for your attention and the question.



What I am calling a manual download is to

1. Generate a list of videos
2. Select one of them, bringing up the "Options for:" dialog
3. Then Selecting "Record" without selecting "Watch now"

This is the method I also use for all the content I record via the UPnP plugin, where I am seeing the same symptom. The commonality is that the both use download (temp?) folders in the SageTV server directory (youtube_downloads and upnp_downloads).

In your YouTube plug-in, the other way to record is to generate the list of videos, Watch one of them, wait for it to complete downloading, then "save downloaded video" (during or after you've watched it).

Watched videos are downloaded to a different "temp" folder than recorded-only videos. The default, for watched videos, is the system temp folder. It may be relevant to note that I have configured the plug-in to use a media folder as my temp directory. Another media folder is the directory where I save the watched videos. No files accumulate in the temp directory with this second option. It seems that only the second option uses the temp directory I configure.

Got it. I will investigate.

[Rob23]

Originally Posted by Rob23

Attached is a logfile in which the out of bounds exception occurs. I believe this issue only occurs if the video length is equal to or greater than 60 minutes.

Tom - Any thoughts on the out of bounds exception?

[tmiranda]

Originally Posted by Rob23

Tom - Any thoughts on the out of bounds exception?

Sorry Rob, I haven't looked at it yet.

[tmiranda]

Originally Posted by egeller

@tmiranda

I have a long-standing issue, I am just now getting to investigating. It affects both the YouTube AND the UPnP plugins.

When I manually record a video, it downloads to the ../sagetv/server/youtube-OR-upnp_downloads folder. At completion it is copied to the folder(s) I specify. However the copy in the "*_downloads" folder remains. I understand that the processes should be doing a MOVE not a COPY.

I have tried a number of things, but cannot see any change in behavior. The log always has a line, at the end of the download process that look like this:

For now, I am manually clearing the folders weekly, more or less. However, do you have any thoughts on what I might try to get the folders to clear out on their own?

I cannot help thinking that this is an Unraid or Linix issue. I tried modifying directory permissions with no effect. I turned logging to extreme and did not see any additional information. If I, first, watch a video, then "save downloaded video" I can see that the temporary copy disappears. The scope of the current issue is on manually recorded files, using the directories, as noted above.

Attached are the 20 or so lines from the log, where the process is ending.

I think I tracked down this problem. Try using the attached .jar and let me know if it works.

[tmiranda]

Originally Posted by Rob23

Tom - Any thoughts on the out of bounds exception?

I've watched several videos that were over two hours long and can't reproduce the error. Can you give me an example of a specific video that fails on your system?

On the positive side, I did get to see a few old F1 races

[egeller]

Originally Posted by tmiranda

I think I tracked down this problem. Try using the attached .jar and let me know if it works.

Thanks for your help on this.

A quick test showed that it works, kind of. I need to go through the test in more detail so I can understand and write up what I am seeing. Give me a couple of days and I will get back to you.

The quick answer is that I saw it work, and I also got it to fail, but I need to review what I did.

[egeller]

Well, you are on the right track. It works OK the first time after a restart. It does not necessarily work the second time.

Attached is a log and a screen print of a test where download-1 worked and download-2 failed to copy to the final location and remained in ..\sagetv\server\youtube_downloads.

The first download is YT file "fEjag-a7TyU" and it was moved and renamed to "Duston_Ward-Georgia_Dome_Implosion...."

The second file is identified as "IYlkHs19Vu4". It stayed in the youtube_downloads folder.

Let me know what you think.

[tmiranda]

Originally Posted by egeller

Well, you are on the right track. It works OK the first time after a restart. It does not necessarily work the second time.

Attached is a log and a screen print of a test where download-1 worked and download-2 failed to copy to the final location and remained in ..\sagetv\server\youtube_downloads.

The first download is YT file "fEjag-a7TyU" and it was moved and renamed to "Duston_Ward-Georgia_Dome_Implosion...."

The second file is identified as "IYlkHs19Vu4". It stayed in the youtube_downloads folder.

Let me know what you think.

The second video didn't get deleted because of a timing issue that can occur when a download is started, and then checked for completion before it starts to progress. I'll have to fix that.

I don't think this condition will happen too often. Are you seeing many videos not being moved or only a few?

[tmiranda]

Originally Posted by tmiranda

The second video didn't get deleted because of a timing issue that can occur when a download is started, and then checked for completion before it starts to progress. I'll have to fix that.

I don't think this condition will happen too often. Are you seeing many videos not being moved or only a few?

Try this jar. I made a fix and added some debugging code.

[egeller]

Originally Posted by tmiranda

Are you seeing many videos not being moved or only a few?

I am seeing only the FIRST video being moved. All others are not moving, where I download / record / save a file without watching it.

I took your latest JAR and did some testing. Results are attached. Let me know if you have any questions.

There is a full SageTV log, in two parts, as well as an RTF file describing the testing steps I took.

To confirm "save downloaded file" still works and has no impact on what we're debugging, I included a couple of tests for that too. There are 4 tries of the manual / did-not-watch download. The first try moved OK. The other three did not move.

[tmiranda]

Originally Posted by egeller

I am seeing only the FIRST video being moved. All others are not moving, where I download / record / save a file without watching it.

I took your latest JAR and did some testing. Results are attached. Let me know if you have any questions.

There is a full SageTV log, in two parts, as well as an RTF file describing the testing steps I took.

To confirm "save downloaded file" still works and has no impact on what we're debugging, I included a couple of tests for that too. There are 4 tries of the manual / did-not-watch download. The first try moved OK. The other three did not move.

OK, I'll look at this next week. I'm out of town at the moment.

[tmiranda]

Originally Posted by egeller

I am seeing only the FIRST video being moved. All others are not moving, where I download / record / save a file without watching it.

I took your latest JAR and did some testing. Results are attached. Let me know if you have any questions.

There is a full SageTV log, in two parts, as well as an RTF file describing the testing steps I took.

To confirm "save downloaded file" still works and has no impact on what we're debugging, I included a couple of tests for that too. There are 4 tries of the manual / did-not-watch download. The first try moved OK. The other three did not move.

It's the same problem, my fix didn't work. Let's try this again - new jar attached.

[egeller]

Originally Posted by tmiranda

It's the same problem, my fix didn't work. Let's try this again - new jar attached.

Thanks for your help and concern on this.

That did the trick. I could include my testing document, but I am too lazy. Let me know if you want it, and I can send it to you.

Everything worked as expected. Files go into the "youtube" directory, are copied to the final directory, and are deleted from the "youtube" directory. All of the previously seen issues are resolved, as near as I can tell.

Now, one other comment: The PlayOn.JAR has the same issue. Do you want me to post a comment to that thread, or is this comment enough to request a fix there? The truth be told, I noticed it there first, as the file sizes are larger and it became an issue in Unraid Cache drive management.

[tmiranda]

Originally Posted by egeller

Thanks for your help and concern on this.

That did the trick. I could include my testing document, but I am too lazy. Let me know if you want it, and I can send it to you.

Everything worked as expected. Files go into the "youtube" directory, are copied to the final directory, and are deleted from the "youtube" directory. All of the previously seen issues are resolved, as near as I can tell.

Now, one other comment: The PlayOn.JAR has the same issue. Do you want me to post a comment to that thread, or is this comment enough to request a fix there? The truth be told, I noticed it there first, as the file sizes are larger and it became an issue in Unraid Cache drive management.

Great. I would be interested in seeing the log. I want to make sure my fix worked as expected and not by accident

No need to comment on the other thread, I'll update that code when I can. It will be easy now that I have the fix.

[egeller]

Originally Posted by tmiranda

Great. I would be interested in seeing the log. I want to make sure my fix worked as expected and not by accident

Here is the log. Let me know if you need anything more.

[tmiranda]

Originally Posted by egeller

Here is the log. Let me know if you need anything more.

That log looks good. Have you tried downloading longer videos? The videos you posted all downloaded rather quickly, I want to make sure that it works for videos that take 2 or 3 minutes to download.

[egeller]

Originally Posted by tmiranda

Have you tried downloading longer videos? The videos you posted all downloaded rather quickly, I want to make sure that it works for videos that take 2 or 3 minutes to download.

Longer videos also downloaded OK.

[tmiranda]

Originally Posted by egeller

Longer videos also downloaded OK.

Thanks.

[Tiki]

Does this same bug exist in the UPNP plugin for Playon also?

[tmiranda]

Originally Posted by Tiki

Does this same bug exist in the UPNP plugin for Playon also?

Probably.