SageTV Community

SageTV Community (http://forums.sagetv.com/forums/index.php)
-   SageTV v7 Customizations (http://forums.sagetv.com/forums/forumdisplay.php?f=47)
-   -   Plugin: Sage Recording Extender 4 (SREv4) (http://forums.sagetv.com/forums/showthread.php?t=57364)

kchase56 07-29-2019 07:06 AM

Looks like I'm only having an issue with MLB for some reason.
If I use http://localhost:8080/livepvrdata/events?date=20181125, I get the following results which seems to be correct for that date 11/25/2018 =>

COLLEGE BASKETBALL
==================
Florida State vs. Villanova Final
LSU vs. Oklahoma State Final
UAB vs. Canisius Final
Oral Roberts vs. James Madison Final/OT
UNCW vs. E Illinois Final
Duquesne vs. UMass Lowell Final
Cent Michigan vs. Siena Heights Final
Morgan State vs. Mt. St. Mary's Final
Utah vs. Northwestern Final
Fordham vs. Alabama A&M Final
Liberty vs. Savannah State Final
Illinois vs. Miss Valley St Final
Vermont vs. Bucknell Final
Quinnipiac vs. Maine Final
Columbia vs. St Joseph's BKN Final
Morehead State vs. Lipscomb Final
FAU vs. FGCU Final
SIU-E vs. Incarnate Word Final
Southern Miss vs. William Carey Final
Fairfield vs. Denver Final
Oakland vs. N Illinois Final
Bowling Green vs. Drexel Final
Fair. Dickinson vs. Lafayette Final
Wichita State vs. Rice Final
Gardner-Webb vs. Arkansas State Final
UNC Asheville vs. Furman Final
Cent Conn St vs. Pine Manor Final
Hawai'i vs. Fresno State Final
Long Beach St vs. Oregon State Final
Cleveland State vs. Purdue FW Final
SF Austin vs. St. Edward's Final
Seattle vs. Longwood Final
Richmond vs. Hampton Final
Brown vs. Bryant Final
Towson vs. Loyola MD Final
Charleston vs. Memphis Final
USC vs. CSU Bakersfield Final
Samford vs. SC State Final
Grand Canyon vs. La Salle Final
Seton Hall vs. Miami Final

COLLEGE FOOTBALL
================
No games for this date.

MLB BASEBALL
============
No games for this date.

FORMULA ONE RACING
==================
Abu Dhabi Grand Prix Race Complete

NASCAR RACING
=============
No games for this date.

NBA BASKETBALL
==============
Lakers vs. Magic Final
Pistons vs. Suns Final
Raptors vs. Heat Final
Grizzlies vs. Knicks Final
Hawks vs. Hornets Final
Nets vs. 76ers Final
Kings vs. Jazz Final
Trail Blazers vs. Clippers Final

NFL FOOTBALL
============
Bills vs. Jaguars Final
Bengals vs. Browns Final
Jets vs. Patriots Final
Eagles vs. Giants Final
Buccaneers vs. 49ers Final
Panthers vs. Seahawks Final
Ravens vs. Raiders Final
Chargers vs. Cardinals Final
Broncos vs. Steelers Final
Colts vs. Dolphins Final
Vikings vs. Packers Final

NHL HOCKEY
==========
Coyotes vs. Flames Final
Lightning vs. Devils Final
Predators vs. Ducks Final
Kings vs. Oilers Final

COLLEGE SOFTBALL
================
No games for this date.

COLLEGE BASEBALL
================
No games for this date.

WOMEN'S COLLEGE BASKETBALL
==========================
Hawai'i vs. Stanford Final
Fordham vs. Texas Final
Duke vs. Missouri Final
Miami vs. Temple Final
Richmond vs. Virginia Tech Final
Ball State vs. Quinnipiac Final
Bowling Green vs. Mercer Final
W Carolina vs. North Alabama Final
Rider vs. Boston College Final
Florida A&M vs. Omaha Final
Radford vs. Nebraska Final
Fair. Dickinson vs. Howard Final
Sacred Heart vs. Monmouth Final
Brown vs. Bucknell Final
Navy vs. NC A&T Final
Hofstra vs. Stetson Final
New Hampshire vs. Bryant Final
Cent Michigan vs. Chattanooga Final
IUPUI vs. VCU Final
Seton Hall vs. St. Francis BKN Final
UCF vs. Villanova Final
Saint Joseph's vs. Loyola MD Final
Norfolk State vs. Liberty Final
Tenn Tech vs. Old Dominion Final
FIU vs. Campbell Final
C. Carolina vs. Memphis Final
Appalachian St vs. Marshall Final
Butler vs. Illinois State Final
Ga Southern vs. Bethune-Cookman Final
St. Francis PA vs. Duquesne Final
UNC Greensboro vs. NC Pembroke Final
UNC Asheville vs. Wofford Final
Penn State vs. Stony Brook Final
James Madison vs. Winthrop Final
Binghamton vs. Army Final
Colgate vs. Columbia Final
Albany vs. Holy Cross Final
NJIT vs. SE Missouri St Final
Indiana vs. N Illinois Final
Colorado vs. Utah State Final
Colorado State vs. Cornell Final
Northwestern vs. UT Martin Final
Kansas vs. Iona Final
UT Rio Grande vs. St Thomas Hou Final
Auburn vs. Louisiana Final
LSU vs. Little Rock Final
Vanderbilt vs. Presbyterian Final
UMass vs. Fairfield Final
New Orleans vs. UL Monroe Final
Rice vs. McNeese Final
UTSA vs. Weber State Final
UCSB vs. Dartmouth Final
Maine vs. Penn Final
Wright State vs. Marist Final
Virginia vs. Saint Louis Final
Charleston So vs. Jacksonville Final
Lehigh vs. Charlotte Final
American vs. FGCU Final
Washington vs. Michigan Final
San Francisco vs. UT Arlington Final
Nevada vs. USC Final
UNLV vs. Middle Tenn Final
Pittsburgh vs. Tennessee State Final
Oakland vs. Canisius Final
Wisconsin vs. Arkansas Final

Any information on how to proceed debugging this issue would be greatly appreciated. I do have C++ programming experience, but haven't dealt with Java at all.

Thanks.

kchase56 07-29-2019 02:05 PM

The following line in EventMonitorMLB.groovy =>

static private final String FEED_URL = 'http://mlb.mlb.com/gdcross/components/game/mlb/year_%s/month_%s/day_%s/master_scoreboard.json'

needs to be changed to =>

static private final String FEED_URL = 'http://gd.mlb.com/components/game/mlb/year_%s/month_%s/day_%s/master_scoreboard.json'

Making this change locally and building a new livepvrdata.war file gives me correct results.

emveepee 07-29-2019 04:46 PM

Ah ok, that would be an ipv6 vs ipv4 issue.

Martin

skiingwiz 07-30-2019 07:06 PM

Quote:

Originally Posted by kchase56 (Post 621628)
The following line in EventMonitorMLB.groovy =>

static private final String FEED_URL = 'http://mlb.mlb.com/gdcross/components/game/mlb/year_%s/month_%s/day_%s/master_scoreboard.json'

needs to be changed to =>

static private final String FEED_URL = 'http://gd.mlb.com/components/game/mlb/year_%s/month_%s/day_%s/master_scoreboard.json'

Making this change locally and building a new livepvrdata.war file gives me correct results.


When I go to the former, it redirects me to the latter, so there should be no difference.

Can you try going to http://mlb.mlb.com/gdcross/component...coreboard.json in your browser? It should redirect you to http://gd.mlb.com/components/game/ml...coreboard.json and return JSON data.

As with the others here, I'm not having any issues, so maybe it's something specific to your system. I even successfully recorded your Mets, since they were beating up on my Pirates.


Edit: Looking at the DNS, mlb.mlb.com routes through akamai whereas gd.mlb.com goes through cloudfront. Are you doing something to block akamai perhaps?

kchase56 07-30-2019 10:26 PM

Quote:

Originally Posted by skiingwiz (Post 621648)
When I go to the former, it redirects me to the latter, so there should be no difference.

Can you try going to http://mlb.mlb.com/gdcross/component...coreboard.json in your browser? It should redirect you to http://gd.mlb.com/components/game/ml...coreboard.json and return JSON data.

As with the others here, I'm not having any issues, so maybe it's something specific to your system. I even successfully recorded your Mets, since they were beating up on my Pirates.


Edit: Looking at the DNS, mlb.mlb.com routes through akamai whereas gd.mlb.com goes through cloudfront. Are you doing something to block akamai perhaps?

It does redirect me when using my browser.

I've reverted my changes locally and I've added some println statements and the following println =>

static final Event[] getFeed(Date date) throws IOException {
String dateStr = date.format('yyyy|MM|dd', FEED_TIME_ZONE)
String key = "mlbcom_$dateStr"
def events = AppRuntime.instance.statusCache.get(key)
if(events == null) {
def input = dateStr.split('\\|')
try {
def data = new URL(String.format(FEED_URL, input[0], input[1], input[2])).text
println "data = " + data


Produces the following in sagetv_0.txt =>

Wed 7/31 0:07:53.960 data = <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="http://gd.mlb.com/components/game/mlb/year_2019/month_07/day_31/master_scoreboard.json">here</a>.</p>
</body></html>



Does this help at all?

skiingwiz 08-01-2019 08:16 PM

Quote:

Originally Posted by kchase56 (Post 621649)
It does redirect me when using my browser.

I've reverted my changes locally and I've added some println statements and the following println =>

static final Event[] getFeed(Date date) throws IOException {
String dateStr = date.format('yyyy|MM|dd', FEED_TIME_ZONE)
String key = "mlbcom_$dateStr"
def events = AppRuntime.instance.statusCache.get(key)
if(events == null) {
def input = dateStr.split('\\|')
try {
def data = new URL(String.format(FEED_URL, input[0], input[1], input[2])).text
println "data = " + data


Produces the following in sagetv_0.txt =>

Wed 7/31 0:07:53.960 data = <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="http://gd.mlb.com/components/game/mlb/year_2019/month_07/day_31/master_scoreboard.json">here</a>.</p>
</body></html>



Does this help at all?


That confuses me. According to the documentation, the Groovy URL object uses HttpURLConnection which uses the System Property http.maxRedirects which defaults to 20.

All of this matches up with what I see on my system. I can run just the URL line and get the correct answer, following the redirects.

Are you behind any kind of proxy, or something like that?

kchase56 08-01-2019 09:12 PM

Quote:

Originally Posted by skiingwiz (Post 621679)
That confuses me. According to the documentation, the Groovy URL object uses HttpURLConnection which uses the System Property http.maxRedirects which defaults to 20.

All of this matches up with what I see on my system. I can run just the URL line and get the correct answer, following the redirects.

Are you behind any kind of proxy, or something like that?

I am not behind any kind of proxy.

kchase56 08-02-2019 10:46 AM

1 Attachment(s)
If I run the following groovy script on the server machine, it works perfectly.
Could running the script through the Jetty server be causing this issue?

skiingwiz 08-12-2019 08:35 PM

While investigating this issue, I discovered that the gd.mlb.com data source is deprecated and they are directing everyone to move to a new statsapi.mlb.com with a different API and output format. So this issue really is moot.

I have the update ready to go, but I need some time to test it. I was thinking about not releasing it until after this MLB season is over. But if you'd like to test, you can download the war file here (in a zip). It seems to work fine locally, but I have done very little testing so far.

macsupergeek 08-16-2019 11:30 AM

new sports
 
Any chance the LLWS can be added? Looks like the scores are posted and updated via this website - https://www.littleleague.org/world-s.../world-series/

or maybe even a scrape of this one - looking for it to say final --
https://gc.com/tmt/summer-2019/littl...cb90002195a5da

I guess since its such a temporary and small event, maybe too much work?

skiingwiz 08-19-2019 07:44 PM

Quote:

Originally Posted by macsupergeek (Post 621840)
Any chance the LLWS can be added? Looks like the scores are posted and updated via this website - https://www.littleleague.org/world-s.../world-series/

or maybe even a scrape of this one - looking for it to say final --
https://gc.com/tmt/summer-2019/littl...cb90002195a5da

I guess since its such a temporary and small event, maybe too much work?

I won't have time to put this in place before the end of this year's tournament. I could get it in for next year, but I'll need a source that's better than just scraping a web site. Those are very likely to change and small changes can break the whole thing. All of the other monitored sports are hitting URLs that are returning some sort of structured data (usually JSON, but other formats would be viable, XML, YAML, etc.) If something like that exists for the LLWS, it would be very easy to add.

I'll add this to my list of ideas and see if I can find a data source. If anyone else knows of a source, speak up.

cat6man 03-29-2020 12:17 PM

this is my annual time to sing the praises of the best app EVER, SRE.

this year however, baseball's opening day has been postponed, but i'll note that the one pre-season red sox game i recorded was, in fact, picked up and tracked by SRE. :clap:

stay safe and healthy everyone, and remember to help your neighbors whenever you can.


All times are GMT -6. The time now is 09:18 PM.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.