SageTV Community  

Go Back   SageTV Community > SageTV Development and Customizations > SageTV v7 Customizations

Notices

SageTV v7 Customizations This forums is for discussing and sharing user-created modifications for the SageTV version 7 application created by using the SageTV Studio or through the use of external plugins. Use this forum to discuss plugins for SageTV version 7 and newer.

Reply
 
Thread Tools Search this Thread Display Modes
  #781  
Old 02-23-2020, 06:41 AM
tmiranda's Avatar
tmiranda tmiranda is offline
SageTVaholic
 
Join Date: Jul 2005
Location: Central Florida, USA
Posts: 5,842
The API has hit a quota limit. Since I do not pay, limits are set on the number and type of queries that can be done over certain periods of time. The quota for some of the APIs have been exceeded due to a big spike on 2/21.

I think the error will go away when the usage stats are reset. I don't know exactly when that will be because I've never seen this error before.
__________________

Sage Server: 8th gen Intel based system w/32GB RAM running Ubuntu Linux, HDHomeRun Prime with cable card for recording. Runs headless. Accessed via RD when necessary. Four HD-300 Extenders.

Last edited by tmiranda; 02-23-2020 at 06:47 AM.
Reply With Quote
  #782  
Old 02-24-2020, 10:20 AM
gary201 gary201 is offline
Sage Advanced User
 
Join Date: Dec 2010
Location: Rochester NY
Posts: 115
That's interesting because the same issue is happening with the PlayOn Windows application as well. I can see what I've subscribed to, but nothing is visible.
Reply With Quote
  #783  
Old 02-24-2020, 10:46 AM
jerrym jerrym is offline
Sage User
 
Join Date: Apr 2016
Location: Minnesota
Posts: 37
I also have the same issue, and it started mid morning on the 21st.

Is it possible the transaction count spiked because of some sort of error that sent the code into a retry?

Love the functionality and the hard work that went into developing it.

Is there any additional information can I provide to help with this issue?
Reply With Quote
  #784  
Old 02-25-2020, 10:26 AM
tmiranda's Avatar
tmiranda tmiranda is offline
SageTVaholic
 
Join Date: Jul 2005
Location: Central Florida, USA
Posts: 5,842
Quote:
Originally Posted by jerrym View Post
I also have the same issue, and it started mid morning on the 21st.

Is it possible the transaction count spiked because of some sort of error that sent the code into a retry?

Love the functionality and the hard work that went into developing it.

Is there any additional information can I provide to help with this issue?
I don't really know why it's happening. I will look into it later today or tomorrow. I was hoping it would clear up on its own.....
__________________

Sage Server: 8th gen Intel based system w/32GB RAM running Ubuntu Linux, HDHomeRun Prime with cable card for recording. Runs headless. Accessed via RD when necessary. Four HD-300 Extenders.
Reply With Quote
  #785  
Old 02-26-2020, 11:20 AM
gary201 gary201 is offline
Sage Advanced User
 
Join Date: Dec 2010
Location: Rochester NY
Posts: 115
Who owns the project for the API used? Is it yours or someone out on GitHub or somewhere? I did some searching around and there are comments on the net from the past where the quota needed to be increased. I've no idea whether that applies here or not, but it certainly seems plausible given what you reported earlier.
Reply With Quote
  #786  
Old 02-27-2020, 09:47 AM
tmiranda's Avatar
tmiranda tmiranda is offline
SageTVaholic
 
Join Date: Jul 2005
Location: Central Florida, USA
Posts: 5,842
Quote:
Originally Posted by gary201 View Post
Who owns the project for the API used? Is it yours or someone out on GitHub or somewhere? I did some searching around and there are comments on the net from the past where the quota needed to be increased. I've no idea whether that applies here or not, but it certainly seems plausible given what you reported earlier.
I'm the owner. I suspect you are correct. When I look at the daily quota it is blank. I took a first stab at increasing the quote and failed. It's a complicated form that gets submitted to Google. It seems more geared at a large project rather than the modest needs of my PlugIn. I plan on giving it another shot tonight.

I'm not sure what changed in the first place. Maybe because I haven't done anything with the project for a while?
__________________

Sage Server: 8th gen Intel based system w/32GB RAM running Ubuntu Linux, HDHomeRun Prime with cable card for recording. Runs headless. Accessed via RD when necessary. Four HD-300 Extenders.
Reply With Quote
  #787  
Old 02-27-2020, 03:31 PM
Beaker's Avatar
Beaker Beaker is offline
Sage User
 
Join Date: Oct 2005
Location: Central Iowa
Posts: 70
Maybe this is due to COPPA compliance attempts?
__________________
"Worry does not empty tomorrow of its sorrow; it empties today of its strength." Corrie Ten Boom

Core-i5 6600k
16GB RAM
Hauppauge WinTV-QuadHD
Win10
Reply With Quote
  #788  
Old 02-28-2020, 05:53 AM
tmiranda's Avatar
tmiranda tmiranda is offline
SageTVaholic
 
Join Date: Jul 2005
Location: Central Florida, USA
Posts: 5,842
I just updated the quota limit. Let me know if it's working or not.
__________________

Sage Server: 8th gen Intel based system w/32GB RAM running Ubuntu Linux, HDHomeRun Prime with cable card for recording. Runs headless. Accessed via RD when necessary. Four HD-300 Extenders.
Reply With Quote
  #789  
Old 02-28-2020, 09:51 AM
jerrym jerrym is offline
Sage User
 
Join Date: Apr 2016
Location: Minnesota
Posts: 37
Quote:
Originally Posted by tmiranda View Post
I just updated the quota limit. Let me know if it's working or not.
It does not appear to be. I am still getting the same "There are no new Videos." or "There are no favorite channels" for the menu items and the downloading is not working.

From the Log file:

Fri 2/28 9:41:47.345 [Timer-3@3de4e07d] YT: Subscriptions.getSubscriptionsForAuthorizedUser: JSON error. 403 : Access Not Configured. YouTube Data API has not been used in project 679168243199 before or it is disabled. Enable it by visiting https://console.developers.google.co...t=679168243199 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
Reply With Quote
  #790  
Old 02-28-2020, 01:55 PM
timzool timzool is offline
New Member
 
Join Date: Feb 2020
Location: boaston, ma.
Posts: 2
This was annoying me so I started doing some of my own research. I went over to Google developers and set up my own API key. You see, after some poking around I discovered that this plugin does not hard code the API Keys it uses. The API Keys are a folder named YouTube.properties in the root stageTV directory. You can create your own key, it's free with limitations of course. Most notably being the quota limit that's causing all the problems here. I was up and running in moments but I made some disturbing discoveries. I really don't think any of your Solutions are going to work. The YouTube data API has a quota limit of 10,000 queries for a free account. A routine download of my favorite channels videos amounted to over a thousand queries. I don't know if this is normal behavior for the Plug-In or if this represents a change of its interaction with the API. You really need to check on this. Also, I would like to know if this thing should be making any queries when it's not in use. Anyways, with multiple users on a shared API key I don't see how this thing could really function at all. However, until recently it was working just fine so I don't know if something changed. By the way, I'm just using an API key. I'm not bothering to use an OAuth 2.0 key because I discovered that that's only needed if you intend to log into YouTube. I never use this plug-in that way so I don't need it. I changed my download check frequency to once every 24 hours. for the next couple of days I'll keep track of how many queries SageTV makes to see if this works all right.
Reply With Quote
  #791  
Old 03-01-2020, 08:17 PM
Beaker's Avatar
Beaker Beaker is offline
Sage User
 
Join Date: Oct 2005
Location: Central Iowa
Posts: 70
Thanks for looking in to this! It has become my most used way of watching any video media.

For what it's worth I'm still seeing the same behavior and error in logs as jerrym.
__________________
"Worry does not empty tomorrow of its sorrow; it empties today of its strength." Corrie Ten Boom

Core-i5 6600k
16GB RAM
Hauppauge WinTV-QuadHD
Win10
Reply With Quote
  #792  
Old 03-02-2020, 03:54 AM
Jaroch76 Jaroch76 is offline
Sage User
 
Join Date: Apr 2015
Posts: 11
Quote:
Originally Posted by timzool View Post
This was annoying me so I started doing some of my own research. I went over to Google developers and set up my own API key. You see, after some poking around I discovered that this plugin does not hard code the API Keys it uses. The API Keys are a folder named YouTube.properties in the root stageTV directory. You can create your own key, it's free with limitations of course. Most notably being the quota limit that's causing all the problems here. I was up and running in moments but I made some disturbing discoveries. I really don't think any of your Solutions are going to work. The YouTube data API has a quota limit of 10,000 queries for a free account. A routine download of my favorite channels videos amounted to over a thousand queries. I don't know if this is normal behavior for the Plug-In or if this represents a change of its interaction with the API. You really need to check on this. Also, I would like to know if this thing should be making any queries when it's not in use. Anyways, with multiple users on a shared API key I don't see how this thing could really function at all. However, until recently it was working just fine so I don't know if something changed. By the way, I'm just using an API key. I'm not bothering to use an OAuth 2.0 key because I discovered that that's only needed if you intend to log into YouTube. I never use this plug-in that way so I don't need it. I changed my download check frequency to once every 24 hours. for the next couple of days I'll keep track of how many queries SageTV makes to see if this works all right.
Thank you for the research; it would never have occurred to me that the API key wasn't hard-coded.

Like many others, this plugin has become a key part of my video-watching experience. As far as I'm concerned, if this means generating my own API key, I'm fine with that.

Also, in case I forget to say so later, thank you tmiranda. Your plugin is wonderful.
Reply With Quote
  #793  
Old 03-02-2020, 02:56 PM
tmiranda's Avatar
tmiranda tmiranda is offline
SageTVaholic
 
Join Date: Jul 2005
Location: Central Florida, USA
Posts: 5,842
Quote:
Originally Posted by timzool View Post
This was annoying me so I started doing some of my own research. I went over to Google developers and set up my own API key. You see, after some poking around I discovered that this plugin does not hard code the API Keys it uses. The API Keys are a folder named YouTube.properties in the root stageTV directory. You can create your own key, it's free with limitations of course. Most notably being the quota limit that's causing all the problems here. I was up and running in moments but I made some disturbing discoveries. I really don't think any of your Solutions are going to work. The YouTube data API has a quota limit of 10,000 queries for a free account. A routine download of my favorite channels videos amounted to over a thousand queries. I don't know if this is normal behavior for the Plug-In or if this represents a change of its interaction with the API. You really need to check on this. Also, I would like to know if this thing should be making any queries when it's not in use. Anyways, with multiple users on a shared API key I don't see how this thing could really function at all. However, until recently it was working just fine so I don't know if something changed. By the way, I'm just using an API key. I'm not bothering to use an OAuth 2.0 key because I discovered that that's only needed if you intend to log into YouTube. I never use this plug-in that way so I don't need it. I changed my download check frequency to once every 24 hours. for the next couple of days I'll keep track of how many queries SageTV makes to see if this works all right.
I knew from the beginning that a shared (free) key would not be suitable for heavy use. It did work fine for the limited use needed by the plugin. They must have changed something but I sure can't see what. My current quota is at 0 per day and I can't seem to increase it

Generating your own key as you described will work. That's the main reason I put the key in the properties file. (Clearly, this would not be a good thing to do if the plugin was a "real" project.)
__________________

Sage Server: 8th gen Intel based system w/32GB RAM running Ubuntu Linux, HDHomeRun Prime with cable card for recording. Runs headless. Accessed via RD when necessary. Four HD-300 Extenders.
Reply With Quote
  #794  
Old 03-02-2020, 03:34 PM
gary201 gary201 is offline
Sage Advanced User
 
Join Date: Dec 2010
Location: Rochester NY
Posts: 115
Is that something you think we're all going to need to do (create our own keys)?
Reply With Quote
  #795  
Old 03-02-2020, 04:07 PM
jerrym jerrym is offline
Sage User
 
Join Date: Apr 2016
Location: Minnesota
Posts: 37
I tried to create my own key and put it in the properties file, but it is not working for me. I am not getting the error message in log file anymore.

I do login to YouTube, so that might be a difference.

Could someone help with some instructions on how to properly create and configure the API key?
Reply With Quote
  #796  
Old 03-03-2020, 03:53 PM
timzool timzool is offline
New Member
 
Join Date: Feb 2020
Location: boaston, ma.
Posts: 2
Quote:
Originally Posted by jerrym View Post
I tried to create my own key and put it in the properties file, but it is not working for me. I am not getting the error message in log file anymore.

I do login to YouTube, so that might be a difference.

Could someone help with some instructions on how to properly create and configure the API key?
It's not enough just to create a key. First you need to create a project. Then you need to add the "YouTube Data API v3" to your project. Now, I don't log into YouTube, so, I don't know what else is required for that. There are a couple more YouTube apis. All that I'm using is the "YouTube Data API v3". You'll have to ask timiranda if there is anything else required for logging in. When you create your project, it will be created with a whole bunch of apis that you don't need. From the menu go to home, then go to apis overview. On the bottom of the screen they will all be listed. You can just disable them one by one. Add the YouTube data API by clicking on apis and services above. Then you simply find it and add it. Then of course, you have to create your API key. On the menu it's under apis and services/credentials. And if you intend to log into YouTube, You'll also need the OAuth 2.0 client ID and client secret keys. Again, if this comes with a need for other apis I don't know.
Reply With Quote
  #797  
Old 03-05-2020, 04:51 AM
Jaroch76 Jaroch76 is offline
Sage User
 
Join Date: Apr 2015
Posts: 11
Success!

I finally sat down and created my own API key this morning, and it seems to be working. Thank you to everyone who has put hard work into building this wonderful program and debugging the latest issue.

For those who having trouble navigating the less-than-straightforward process of creating an API key, here's what I had to do:
  1. In my web browser, I went to https://developers.google.com/youtub...etting-started. This gave me the basic instructions I needed for starting.
  2. Following their instructions, I opened the Google Developers Console.
  3. I followed their prompts to create a new project, and attached the YouTube Data API v3 API to this project.
  4. I then navigated to the Credentials section of the API dashboard.
  5. I created an API key and an OAuth 2.0 Client ID.
  6. I navigated to the OAuth consent screen section and set up a consent screen. I don't know if this was needed; Google said I should do it.
  7. I then opened the directory containing SageTV and the YouTube-DL configuration file.
  8. After backing up the configuration file, I edited it, putting my new API Key, OAuth Key, and OAuth Client Secret in the respective fields.
  9. Then I loaded SageTV, and logged out of YouTube.
  10. I logged back into YouTube, and used the URL and Device Code provided by SageTV to tell Google that, yes, SageTV was allowed to connect to YouTube.
  11. Somewhere along the lines, I got a security alert email from Google. I followed the link in this email to acknowledge the alert and make Google happy.

I may have missed something. But those are the important things.
Reply With Quote
  #798  
Old 03-05-2020, 06:06 AM
Jaroch76 Jaroch76 is offline
Sage User
 
Join Date: Apr 2015
Posts: 11
So much for success.

I created my own API key earlier this morning. For about twenty minutes, it worked.

But within that short span, the quota has already reached 9756 queries.

Now, the log file is full of lines like these:
Quote:
Thu 3/5 6:20:06.282 [Timer-1@32d108dd] YT: Search.getNextPage: JSON error. 403 : Daily Limit Exceeded. The quota will be reset at midnight Pacific Time (PT). You may monitor your quota usage and adjust limits in the API Console: https://console.developers.google.co...t=258704486810
More worrisome, after a few minutes, the quota-errors went away to be replaced by these:
Quote:
Thu 3/5 6:21:36.775 [Timer-1@32d108dd] YT: Search.getNextPage: JSON error. 401 : Invalid Credentials
The error messages were being written to the log file at a rate of several times per second. I eventually had to restart the SageTV service to stop the runaway spamming.

Also, during the short period that it worked, it downloaded five videos... but downloaded them to the "youtube_downloads" subdirectory in the SageTV directory, not the directory where they're configured to go.

I don't know if it'll help, but I've attached the log file for that time period.
Attached Files
File Type: zip sagetv_1.zip (191.6 KB, 7 views)
Reply With Quote
  #799  
Old 03-06-2020, 11:04 PM
robogeek robogeek is offline
Sage Expert
 
Join Date: Apr 2003
Location: Appleton, WI USA
Posts: 566
Just to add to this, in addition to using my own YouTube Data API key, oAuth key, and oAuth Client Secret, I also went into the YouTube (2015) options menu to change a few settings to prevent it from blowing through the laughably low 10000 unit quota limit that someone at YouTube actually thought was a good idea.

For full disclosure, I don't use the YouTube (2015) plugin for browsing or searching for YouTube content. I only use the Browse Favorite Channels menu item to manage my channel favorites when needed. I don't use the What's New, Recommended For You, My YouTube Channel, Best of YouTube, or any of those other menu items in the plugin. I suspect using those other menu items will quickly blow through the 10000 unit quota limit. I use the YouTube (2015) plugin strictly for recording new videos from 25 different YouTube channels (and averaging less than 4000 of the 10000 quota units per day) and have not had any trouble since making the following additional changes...

YouTube Menu Options:
- YouTube Menu Options >> Maximum Items to Fetch
I set Maximum Items to Fetch to the lowest number, 50. I believe the YouTube Data API still returns results in sets of up to 50 with a specific quota usage per 50 results. That is, a search request that returns 1 result would have the same quota usage as a search request that returns 50 results. A request that returns 51 to 100 results would have double the quota usage of a request that returns 1 to 50 results.

- YouTube Menu Options >> Recording Options >> Favorite Check Interval
I set this to 24 so it only checks for favorites once per day. Checking two or three times per day (or more) will result in two or three times (or more) the quota usage that pertains to checking for new videos on your channel favorites.

- YouTube Menu Options >> Advanced Options >> Include Keyword Favs in "What's New"
I set this option to Disabled. I'm not sure if it helps, but anything that looks like it may generate quota usage should be disabled if it is not absolutely needed.

For each of my YouTube channel favorites, these are the Favorite Options:
- Fetch Watched Videos: Disabled
- Include in "What's New": Disabled
- Record New Episodes: Enabled
- Get Episodes Since/From: 'A Week Ago' **
- Re-Record Deleted Episodes: No
- Minimum Duration: Any Duration
- Simulate (Do not record): Disabled

** I'm not sure how the plugin code is written regarding the API requests for new episodes, so it may be necessary to set this option to 'A Week Ago' on a regular basis...maybe once a month or once every few months, etc... depending on how many channel favorites are set and how often videos are released on the channel favorites. If the daily quota usage slowly creeps up over time, resetting this setting to 'A Week Ago' again may bring quota usage back down.
__________________
--Jason

Server Hardware: GIGABYTE GA-EP45-UD3R, Intel Q9550 CPU 2.83GHz, 11GB RAM, 1xHDHR, 1xHVR1600, 1xHVR2250
29TB Server Storage: 1TB SSD (OS), 1TB (data), 2x6TB+2x10TB (22TB FlexRaid storage pool), 2x2TB (recordings), 1x750GB (VMs).
Server Software: Win10 Pro x64 OS, SageTV 64bit v9.2.0.441, Java 1.8 u241, PlayOn, Comskip (Donator) v0.82.003, WampServer v2.5.
Clients: 3xHD300s, 2xHD100, 2xPlaceshifters
Reply With Quote
  #800  
Old 03-07-2020, 07:37 AM
gary201 gary201 is offline
Sage Advanced User
 
Join Date: Dec 2010
Location: Rochester NY
Posts: 115
I can't help but wonder what other PVR implementations are doing for YouTube. Are they equally affected or do they have a completely different implementation so that they are unaffected by whatever change Google has done here? Or is it the case that all PVR solutions will require each user to create their own projects, keys tokens, etc.?

One interesting coincidence is that PlayOn may similarly be affected. I similarly noticed that the PlanOn Windows app can't see that anything exists for any of my subscribed YouTube channels. The folks at PlayOn insist they can't reproduce this, and I've insisted that I'm not going to give them my Google credentials for further debugging (for obvious reasons since that gives them access to email, and other things authenticated by Google). Is it possible that the YouTube plugin for Sage could be affecting the ability for PlayOn to also see that same content?
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Plex Hacked July 1, 2015 tvmaster2 General Discussion 5 07-08-2015 02:43 AM
Will 2015 leap second effect SageTV? Paul H SageTV Software 10 01-16-2015 07:43 AM
Merry Christmas & Happy New Year 2015 SHS General Discussion 25 01-01-2015 08:59 PM
Plugin request for SageTV Windows Client: Autofrequency plugin mkanet SageTV v7 Customizations 3 11-12-2011 09:33 AM
MediaPlayer Plugin/STV Import: Winamp Media Player Plugin deria SageTV Customizations 447 12-11-2010 07:38 PM


All times are GMT -6. The time now is 11:09 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.