SageTV Community  

Go Back   SageTV Community > SageTV Development and Customizations > SageTV v7 Customizations

Notices

SageTV v7 Customizations This forums is for discussing and sharing user-created modifications for the SageTV version 7 application created by using the SageTV Studio or through the use of external plugins. Use this forum to discuss plugins for SageTV version 7 and newer.

Reply
 
Thread Tools Search this Thread Display Modes
  #801  
Old 03-07-2020, 04:19 PM
robogeek robogeek is offline
Sage Expert
 
Join Date: Apr 2003
Location: Appleton, WI USA
Posts: 566
Google API projects created prior to May 2016 and using the YouTube Data API had a very high default quota limit of 50 million quota units per day. I believe those projects were allowed to retain their high quota limits even after the default quota limits for new projects were lowered to 1 million per day in May 2016, and when lowered again down to 10 thousand per day starting in January 2019. My best guess would be Google/YouTube is now finally invalidating these older high quota projects, resetting all older high quota projects to the new default 10k limits, or zeroing the quota to force the developers with older projects to create a new project with the new lower default 10k quota limit.

The YouTube addon for Kodi was also affected. I'm sure others using a shared key for the YouTube Data API have been affected as well and will require end-users to create their own YouTube API keys going forward. Apps that use website scrapers obviously won't be affected by API quota limits. I could be wrong, but I suspect the YouTube channel for PlayOn is using a website scraper to gather its data from YouTube and not using the YouTube Data API.

@gary201, for PlayOn, I just checked mine and I still have full YouTube access on my PlayOn installation, but I had a similar issue with the AT&T TV Now channel. It loaded up in PlayOn, but wouldn't list any videos. The PlayOn support solution was to delete the MediaMall cache directory, restart the MediaMall service, wait a full 60 minutes after restarting the service, and then try using the AT&T TV Now channel again. If you want to try that cache clearing trick to see if it fixes your YouTube issue, here's the email I got from support:
Quote:
Hi Jason,

Thank you for your patience. .

As a test, please navigate to the following location and delete the Cache folder contained within:

%PROGRAMDATA%\MediaMall\Cef

After doing so, stop and start the PlayOn server, wait a full 60 minutes and then try again.

Let us know if there's any difference.
Best regards
__________________
--Jason

Server Hardware: GIGABYTE GA-EP45-UD3R, Intel Q9550 CPU 2.83GHz, 11GB RAM, 1xHDHR, 1xHVR1600, 1xHVR2250
29TB Server Storage: 1TB SSD (OS), 1TB (data), 2x6TB+2x10TB (22TB FlexRaid storage pool), 2x2TB (recordings), 1x750GB (VMs).
Server Software: Win10 Pro x64 OS, SageTV 64bit v9.2.0.441, Java 1.8 u241, PlayOn, Comskip (Donator) v0.82.003, WampServer v2.5.
Clients: 3xHD300s, 2xHD100, 2xPlaceshifters
Reply With Quote
  #802  
Old 03-09-2020, 03:58 PM
gary201 gary201 is offline
Sage Advanced User
 
Join Date: Dec 2010
Location: Rochester NY
Posts: 115
I tried what was suggested to you, to delete the cache directory. Unfortunately that didn't solve the problem that I'm seeing. Of the 30 or so subscribed YT channels, PlayOn (the windows app) doesn't see anything to view when you navigate into that subscribed channel.

If everyone is going to have to create their own YT project, keys, etc., I think most will just give up and go back to watching on a browser (and maybe casting it to their TV). I never used the SageTV plugin to record YT channels, just view them on the weekends. What I liked was the ability that the plugin provided to just flag those that I have watched. This probably will be the death of YT plugins.
Reply With Quote
  #803  
Old 03-09-2020, 04:11 PM
gary201 gary201 is offline
Sage Advanced User
 
Join Date: Dec 2010
Location: Rochester NY
Posts: 115
Interesting links for Kodi...

https://howtokodi.eu/tips-and-tricks...uota-exceeded/

https://forum.kodi.tv/showthread.php?tid=267160
Reply With Quote
  #804  
Old 03-10-2020, 09:49 AM
robogeek robogeek is offline
Sage Expert
 
Join Date: Apr 2003
Location: Appleton, WI USA
Posts: 566
Quote:
Originally Posted by gary201 View Post
I tried what was suggested to you, to delete the cache directory. Unfortunately that didn't solve the problem that I'm seeing. Of the 30 or so subscribed YT channels, PlayOn (the windows app) doesn't see anything to view when you navigate into that subscribed channel.
If only the items that are tied to your YouTube account are affected and everything else in the YouTube channel is working, it must be a YouTube login issue. Since Google/YouTube have a more elaborate login than most other websites, you probably need to go through that login process again from within PlayOn Settings, especially now that the PlayOn cache has been cleared.

Try this: Open PlayOn Settings, go the the Channels tab, click the Options button of the YouTube channel, make sure the 'I've authenticated with my youtube credentials' box is checked, then click the Login button to go through the authentication process (you may need to close this authentication window manually when login is completed), then click OK on the YouTube Options box, and click Apply on the PlayOn Settings window. And, just for good measure, I would go back to the General tab and click the Stop button to stop the PlayOn Media Server, wait for it to stop, and click Start to restart it. Give it at least a couple of minutes to start up and then try the YouTube channel again and see if subscribed channels are working.

If that still doesn't work, the only thing left I can think of would be to open both the Internet Explorer and Edge web browsers and delete the cookies and cache of both browsers. Then go back and delete the PlayOn cache again, and restart the PlayOn Media Server from PlayOn Settings (or restart MediaMall Server in Windows Service control panel). Then try logging into YouTube from both the Internet Explorer and Edge web browsers. If logging in via both web browsers is successful, go back into PlayOn Settings and go through the Login procedure for the YouTube channel again. If something is causing the login to fail in either web browser, that probably needs to be investigated and fixed before trying to authenticate via PlayOn Settings YouTube channel.

If you have any Internet ad blocker or other Internet security or Internet privacy related software, plugins, or addons installed on your router, computer, or web browser (only Internet Explorer and Edge), those could be messing with the HTML just enough to cause problems with YouTube in PlayOn. As a test, disable all of them and see if YouTube in PlayOn starts working. If it still doesn't work, then re-enable them. If YouTube in PlayOn starts working after disabling those items, it will probably be difficult to test which one is causing the problem...re-enabling them one-at-a-time while checking YouTube in PlayOn may not work due to caching.

If none of that works, PlayOn Support is probably the only option left to figure out the issue. For PlayOn Support, you could always turn off your two-factor authentication (if it's enabled) and change your password to something else, and then give them your YouTube credentials. Once they figure out what's wrong, change your password again and re-enable two-factor authentication.
__________________
--Jason

Server Hardware: GIGABYTE GA-EP45-UD3R, Intel Q9550 CPU 2.83GHz, 11GB RAM, 1xHDHR, 1xHVR1600, 1xHVR2250
29TB Server Storage: 1TB SSD (OS), 1TB (data), 2x6TB+2x10TB (22TB FlexRaid storage pool), 2x2TB (recordings), 1x750GB (VMs).
Server Software: Win10 Pro x64 OS, SageTV 64bit v9.2.0.441, Java 1.8 u241, PlayOn, Comskip (Donator) v0.82.003, WampServer v2.5.
Clients: 3xHD300s, 2xHD100, 2xPlaceshifters
Reply With Quote
  #805  
Old 03-10-2020, 10:37 AM
gary201 gary201 is offline
Sage Advanced User
 
Join Date: Dec 2010
Location: Rochester NY
Posts: 115
I don't have 2fa enabled. I did go through the login and re-verifying/authorizing when I noticed this (within the PlayOn Setup app). The PLayOn app does see the subscribed YT channels, but they don't show anything. I have a dedicated VM that exists only for the purpose of running the PlayOn app (literally unused for anything else and has been for a couple years now). It's also not (just) my subscribed YT channels, but everything on YT (even the generic 'most popular', 'best of ..', and 'channels' are returning "This Folder is Empty".

I've pretty much given up on YT from anything other than a browser.
Reply With Quote
  #806  
Old 03-10-2020, 12:19 PM
robogeek robogeek is offline
Sage Expert
 
Join Date: Apr 2003
Location: Appleton, WI USA
Posts: 566
Given that information, I would suspect either PlayOn is not up-to-date (current version is v4.5.62) or the operating system or Internet Explorer is not up-to-date and causing issues. Also, since PlayOn is using (I believe) a background Internet Explorer window to handle communications with YouTube, did you try as I suggested above to clear the web browser cookies and cache and try logging in both through the web browser and doing the Login procedure again through PlayOn? This would need to be done within your PlayOn VM.
__________________
--Jason

Server Hardware: GIGABYTE GA-EP45-UD3R, Intel Q9550 CPU 2.83GHz, 11GB RAM, 1xHDHR, 1xHVR1600, 1xHVR2250
29TB Server Storage: 1TB SSD (OS), 1TB (data), 2x6TB+2x10TB (22TB FlexRaid storage pool), 2x2TB (recordings), 1x750GB (VMs).
Server Software: Win10 Pro x64 OS, SageTV 64bit v9.2.0.441, Java 1.8 u241, PlayOn, Comskip (Donator) v0.82.003, WampServer v2.5.
Clients: 3xHD300s, 2xHD100, 2xPlaceshifters
Reply With Quote
  #807  
Old 03-11-2020, 02:20 PM
gary201 gary201 is offline
Sage Advanced User
 
Join Date: Dec 2010
Location: Rochester NY
Posts: 115
That's the PlayOn version installed here, and I did clear the browser history and cookies from both IE and Chrome (they at one time asked I install that).
Reply With Quote
  #808  
Old 03-11-2020, 02:22 PM
jerrym jerrym is offline
Sage User
 
Join Date: Apr 2016
Location: Minnesota
Posts: 37
I followed the procedure in post 797 and it worked to get me up and running. Thanks for the writeup!

I still hit my quota, but it takes about 12 hours now. Still trying to reduce what I am recording and with what frequency it looks for new videos.

One thing I noticed in the Metrics tab for Youtube API is alot of transactions for the youtube.search.list method. This is an expensive method as it has a quota cost of 100 units.

I see it is also using the youtube.channels.list method which has a quota cost of 1 unit. Since I am just recording new videos based on channel favorites, I would expect this to be the most used.

Does anyone know what it uses the youtube.search.list method for? Or if maybe I can change something I am doing to minimize the utilization?
Reply With Quote
  #809  
Old 03-11-2020, 04:01 PM
Jaroch76 Jaroch76 is offline
Sage User
 
Join Date: Apr 2015
Posts: 11
You're Welcome!

Quote:
I followed the procedure in post 797 and it worked to get me up and running. Thanks for the writeup!
You're welcome. I'm glad that I could contribute something to this thread.
Reply With Quote
  #810  
Old 03-12-2020, 04:31 AM
Jaroch76 Jaroch76 is offline
Sage User
 
Join Date: Apr 2015
Posts: 11
Attempt #2: No Go

Quote:
Originally Posted by jerrym View Post
One thing I noticed in the Metrics tab for Youtube API is alot of transactions for the youtube.search.list method. This is an expensive method as it has a quota cost of 100 units.

I see it is also using the youtube.channels.list method which has a quota cost of 1 unit. Since I am just recording new videos based on channel favorites, I would expect this to be the most used.

Does anyone know what it uses the youtube.search.list method for? Or if maybe I can change something I am doing to minimize the utilization?
I reenabled the YouTube plugin for a bit this morning. And hit my quota limit within 5 minutes. I, too, see that the usage of the youtube.search.list method absolutely dwarfs everything else, at roughly 1700 requests, compared to roughly 250 for everything else combined.

Plus, the plugin still saved the few newly downloaded videos to the wrong directory.

Once again, I've attached the log file for today in case it'll help.
Attached Files
File Type: zip sagetv_0.zip (455.3 KB, 3 views)
Reply With Quote
  #811  
Old 03-12-2020, 05:37 PM
Beaker's Avatar
Beaker Beaker is offline
Sage User
 
Join Date: Oct 2005
Location: Central Iowa
Posts: 70
Quote:
Originally Posted by Jaroch76 View Post
[*]After backing up the configuration file, I edited it, putting my new API Key, OAuth Key, and OAuth Client Secret in the respective fields.
Thanks for posting your list, Jaroch76. I'm working my way through this and I'm looking for some clarification. My youtube.properties file does not contain any entry for oauth information. Would you be so kind as to post a copy of yours so that I can see what format is expected? I'm getting the same "There is nothing to display." results like before I plugged in my own API key. My log is showing...
YT: Subscriptions.getSubscriptionsForAuthorizedUser: JSON error. 401 : Invalid Credentials
and I figure this is related to not having any oauth key statement.

I'd guess that particular code string would be sensitive information, so maybe just replace yours with...
GOOGLE OAUTH GIBBERISH HERE
I'm sure I'll get the hint. Thanks for any further help you can offer.
__________________
"Worry does not empty tomorrow of its sorrow; it empties today of its strength." Corrie Ten Boom

Core-i5 6600k
16GB RAM
Hauppauge WinTV-QuadHD
Win10
Reply With Quote
  #812  
Old 03-13-2020, 02:33 AM
Jaroch76 Jaroch76 is offline
Sage User
 
Join Date: Apr 2015
Posts: 11
Quote:
Originally Posted by Beaker View Post
Thanks for posting your list, Jaroch76. I'm working my way through this and I'm looking for some clarification. My youtube.properties file does not contain any entry for oauth information. Would you be so kind as to post a copy of yours so that I can see what format is expected? I'm getting the same "There is nothing to display." results like before I plugged in my own API key. My log is showing...
YT: Subscriptions.getSubscriptionsForAuthorizedUser: JSON error. 401 : Invalid Credentials
and I figure this is related to not having any oauth key statement.

I'd guess that particular code string would be sensitive information, so maybe just replace yours with...
GOOGLE OAUTH GIBBERISH HERE
I'm sure I'll get the hint. Thanks for any further help you can offer.

Beaker: Sure, no problem. You're right that the properties file doesn't explicitly say oauth; it took a bit of trial-and-error on my part to match up what Google said with what the properties file said. Below is my redacted file.

Code:
youtube.apikey=XXXXXXX_xxxx_XXXXXXXXXXXXXXXXXXXXXXXXXX
youtube.clientid=############-XXXXXXXXXXXXXXXXXXXXXX.apps.googleusercontent.com
youtube.clientsecret=XXXXXXXXXXXXXX-XXXXXXXXXXXX
The clientid and clientsecret entries in this file correspond to Google's Client ID and Client Secret information on my API's Credentials page.

I hope that this helps. Good luck.
Reply With Quote
  #813  
Old 03-13-2020, 03:26 PM
Jaroch76 Jaroch76 is offline
Sage User
 
Join Date: Apr 2015
Posts: 11
Quote:
Originally Posted by Jaroch76 View Post
I reenabled the YouTube plugin for a bit this morning. And hit my quota limit within 5 minutes. I, too, see that the usage of the youtube.search.list method absolutely dwarfs everything else, at roughly 1700 requests, compared to roughly 250 for everything else combined.

Plus, the plugin still saved the few newly downloaded videos to the wrong directory.

Once again, I've attached the log file for today in case it'll help.

This time around, after reenabling the YouTube plugin, I configured all but one of my favorite channels to stop automatically downloading new videos. As a result, in the half-hour that the plugin has been enabled, I've only used ~850 quota units, and that number stopped climbing after the first burst of new-video-downloading.

Better still, now that the plugin is downloading videos properly (and I'm not panicking about quota), I see that the videos are being saved in the correct location. I now see that they get downloaded to SageTV\youtube_downloads and then moved elsewhere once fully downloaded.

I'll keep an eye on things and slowly reenable my favorite channels. If I figure out how to reliably avoid massive spikes in quota usage, I'll let you know.
Reply With Quote
  #814  
Old 03-14-2020, 07:40 PM
Beaker's Avatar
Beaker Beaker is offline
Sage User
 
Join Date: Oct 2005
Location: Central Iowa
Posts: 70
Quote:
Originally Posted by Jaroch76 View Post
Beaker: Sure, no problem. You're right that the properties file doesn't explicitly say oauth; it took a bit of trial-and-error on my part to match up what Google said with what the properties file said. Below is my redacted file.

Code:
youtube.apikey=XXXXXXX_xxxx_XXXXXXXXXXXXXXXXXXXXXXXXXX
youtube.clientid=############-XXXXXXXXXXXXXXXXXXXXXX.apps.googleusercontent.com
youtube.clientsecret=XXXXXXXXXXXXXX-XXXXXXXXXXXX
The clientid and clientsecret entries in this file correspond to Google's Client ID and Client Secret information on my API's Credentials page.

I hope that this helps. Good luck.
It did indeed. Thank you.

I hit my quota in about 2 minutes. I'm busy turning things off as fast as I can when it is working. In the mean time, I have started using youtube-dl directly and set it up to work automatically.

I had to key everything in, and will have to do so to add any new channel. I wish I had a way to directly access what was favorited in this plugin, but I couldn't figure it out. I'm not certain, but I think my way (no api usage) moves more bits over the net, but I don't have to worry about google quotas. In my particular case, that would mean this is costing google more than upping the quota. The law of unintended consequences strikes again!
__________________
"Worry does not empty tomorrow of its sorrow; it empties today of its strength." Corrie Ten Boom

Core-i5 6600k
16GB RAM
Hauppauge WinTV-QuadHD
Win10
Reply With Quote
  #815  
Old 03-15-2020, 02:22 PM
Jaroch76 Jaroch76 is offline
Sage User
 
Join Date: Apr 2015
Posts: 11
Quote:
Originally Posted by Jaroch76 View Post
This time around, after reenabling the YouTube plugin, I configured all but one of my favorite channels to stop automatically downloading new videos. As a result, in the half-hour that the plugin has been enabled, I've only used ~850 quota units, and that number stopped climbing after the first burst of new-video-downloading.

Better still, now that the plugin is downloading videos properly (and I'm not panicking about quota), I see that the videos are being saved in the correct location. I now see that they get downloaded to SageTV\youtube_downloads and then moved elsewhere once fully downloaded.

I'll keep an eye on things and slowly reenable my favorite channels. If I figure out how to reliably avoid massive spikes in quota usage, I'll let you know.

I've had the plugin running for two days now without any problems. I believe that I was hitting my API quota so quickly because I had the plugin configured such that the "Get Episodes from/since" option for all of my favorite channels was set to "Very Beginning". And some of my channels have a LOT of videos.

I'm just guessing, but I think that the plugin asks YouTube to return a list of videos since this get-from date, and the size of the returned list affects the API's quota. This theory is backed up by the log files, which indicate that the youtube-channel-query API returns an average of 20 results. Except for that one huge channel of mine, which can return 24,000 results if I let it:

Quote:
Sun 3/15 4:41:20.096 [Thread-397@268ea4ed] YT: Search.searchByChannelId: kind:youtube#searchListResponse etag:"SJZWTG6xR0eGuCOh2bX6w3s4F94/yKHoyTLBeZyIDE2SwK9F4lt7rKQ" nextPageToken:CDIQAA prevPageToken:null totalResults:24167 resultsPerPage:50
For now, I've got just a couple of channels enabled, and they're all configured to only check for videos released within the past month, and this check is only performed once per day. My API usage for the past two days has been around 1500 queries.

As a test, I temporarily reconfigured one of my really large channels to check for all videos from the very beginning and forced the plugin to do the check. That caused a massive spike in Google's query-quota-information graph.

The YouTube plugin doesn't have an option to download videos "within the last week". I can set the get-from date to "a week ago", but that gets translated into 03/08/2020. Still, I can sit down at my computer and periodically reconfigure my favorite channels to use a new get-from date. That's a small price to pay for watching YouTube through SageTV.

I hope that this helps.

Last edited by Jaroch76; 03-15-2020 at 02:30 PM. Reason: I forgot to mention that I had log-file information to back up my theory.
Reply With Quote
  #816  
Old 03-20-2020, 04:15 PM
Jaroch76 Jaroch76 is offline
Sage User
 
Join Date: Apr 2015
Posts: 11
Quote:
Originally Posted by Jaroch76 View Post
The YouTube plugin doesn't have an option to download videos "within the last week". I can set the get-from date to "a week ago", but that gets translated into 03/08/2020. Still, I can sit down at my computer and periodically reconfigure my favorite channels to use a new get-from date. That's a small price to pay for watching YouTube through SageTV.
I've learned yet one more thing. It appears that any change to any setting for any Favorite channel in the YouTube plugin automatically triggers a search for new videos in all of the Favorite channels. I found this out on Wednesday when I sat down to reset the get-from date on all of my channels to a week earlier... and hit my quota fairly quickly.

The secret to this appears to be to go into the SageTV Installed Plugins, disable the YouTube Recording Engine plugin (but not the main YouTube plugin). This allowed me to reset the get-from date on all dozen-or-so channels without triggering a rescan. I then reenabled the YouTube Recording Engine plugin, and all seems well now.
Reply With Quote
  #817  
Old 03-26-2020, 06:41 AM
jerrym jerrym is offline
Sage User
 
Join Date: Apr 2016
Location: Minnesota
Posts: 37
Is there anyway to use multiple sets of credentials with this plugin?

To get around the quota limits, I have created 3 sets of credentials that I rotate through when the plugin gets behind to "catch-up" again. I did this my creating 3 projects, each with a set of credentials, and they I have 3 versions of the youtube.properties file that I rotate and bounce the SageTV service to load in the next set of credentials. I believe you can have up to 10 projects registered.

If the plugin could do this for me, maybe during the background scan, that would be ideal.
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Plex Hacked July 1, 2015 tvmaster2 General Discussion 5 07-08-2015 02:43 AM
Will 2015 leap second effect SageTV? Paul H SageTV Software 10 01-16-2015 07:43 AM
Merry Christmas & Happy New Year 2015 SHS General Discussion 25 01-01-2015 08:59 PM
Plugin request for SageTV Windows Client: Autofrequency plugin mkanet SageTV v7 Customizations 3 11-12-2011 09:33 AM
MediaPlayer Plugin/STV Import: Winamp Media Player Plugin deria SageTV Customizations 447 12-11-2010 07:38 PM


All times are GMT -6. The time now is 08:22 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.